2023-09-06T18:20:08.115 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-06T18:20:08.116 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_ivancich@teuthology --verbose --description rgw/notifications/{beast bluestore-bitmap fixed-2 ignore-pg-availability overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}} --archive /home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243 --name ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi -- /home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243/orig.config.yaml 2023-09-06T18:20:08.158 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-06T18:20:08.252 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243 branch: wip-eric-testing-1 description: rgw/notifications/{beast bluestore-bitmap fixed-2 ignore-pg-availability overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}} email: ivancich@redhat.com first_in_suite: false job_id: '7390243' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-eric-testing-1 ceph: conf: client: debug rgw: 20 rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw lc debug interval: 10 setgroup: ceph setuser: ceph global: osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled sha1: 5535a92bd618b7e580a2fce32e9dfab6620d1086 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 5535a92bd618b7e580a2fce32e9dfab6620d1086 rgw: frontend: beast storage classes: LUKEWARM, FROZEN selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-eric-testing-1 sha1: 5535a92bd618b7e580a2fce32e9dfab6620d1086 owner: scheduled_ivancich@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 831 sha1: 5535a92bd618b7e580a2fce32e9dfab6620d1086 sleep_before_teardown: 0 subset: 1/50 suite: rgw suite_branch: wip-eric-testing-1 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 5535a92bd618b7e580a2fce32e9dfab6620d1086 targets: smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGgXCd8FtdPyuvBoDz1Odlgd52mKpJX9T2D8O3a4AwUgUBoffGlDGApGL+FaQmtbp7NgosmSVVFT864Bf5rUyyM= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBxZTiiM0jxT4QB+tCztuafD3KDXgRBguwmwjNkdFGh/L+PidvLrqhDkcqEGy+D7vC5x6bcz2eMfr5qOlmg9rC4= tasks: - install: null - ceph: null - openssl_keys: null - rgw: client.0: null - rabbitmq: client.0: null - notification-tests: client.0: extra_attr: - amqp_test rgw_server: client.0 - kafka: client.0: kafka_version: 2.6.0 - notification-tests: client.0: extra_attr: - kafka_test rgw_server: client.0 - notification-tests: client.0: rgw_server: client.0 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-06_18:12:39 tube: smithi user: ivancich verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1394 2023-09-06T18:20:08.253 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa; will attempt to use it 2023-09-06T18:20:08.254 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks 2023-09-06T18:20:08.254 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-06T18:20:08.256 INFO:teuthology.task.internal:Checking packages... 2023-09-06T18:20:08.282 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '5535a92bd618b7e580a2fce32e9dfab6620d1086' 2023-09-06T18:20:08.282 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-06T18:20:08.283 INFO:teuthology.packaging:ref: None 2023-09-06T18:20:08.283 INFO:teuthology.packaging:tag: None 2023-09-06T18:20:08.283 INFO:teuthology.packaging:branch: wip-eric-testing-1 2023-09-06T18:20:08.283 INFO:teuthology.packaging:sha1: 5535a92bd618b7e580a2fce32e9dfab6620d1086 2023-09-06T18:20:08.284 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-eric-testing-1 2023-09-06T18:20:08.431 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5801.g5535a92b 2023-09-06T18:20:08.431 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-06T18:20:08.438 INFO:teuthology.task.internal:no buildpackages task found 2023-09-06T18:20:08.439 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-06T18:20:08.449 INFO:teuthology.task.internal:Saving configuration 2023-09-06T18:20:08.463 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-06T18:20:08.505 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-06T18:20:08.544 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-06 18:13:27.778052', 'locked_by': 'scheduled_ivancich@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGgXCd8FtdPyuvBoDz1Odlgd52mKpJX9T2D8O3a4AwUgUBoffGlDGApGL+FaQmtbp7NgosmSVVFT864Bf5rUyyM='} 2023-09-06T18:20:08.576 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-06 18:13:27.776348', 'locked_by': 'scheduled_ivancich@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBxZTiiM0jxT4QB+tCztuafD3KDXgRBguwmwjNkdFGh/L+PidvLrqhDkcqEGy+D7vC5x6bcz2eMfr5qOlmg9rC4='} 2023-09-06T18:20:08.577 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-06T18:20:08.585 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-09-06T18:20:08.586 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-09-06T18:20:08.586 INFO:teuthology.run_tasks:Running task console_log... 2023-09-06T18:20:08.677 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3f0f744a60>, signals=[15]) 2023-09-06T18:20:08.679 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-06T18:20:08.686 INFO:teuthology.task.internal:Opening connections... 2023-09-06T18:20:08.686 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2023-09-06T18:20:08.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-06T18:20:08.767 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2023-09-06T18:20:08.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-06T18:20:08.845 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-06T18:20:08.854 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2023-09-06T18:20:08.878 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2023-09-06T18:20:08.878 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2023-09-06T18:20:08.939 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2023-09-06T18:20:08.939 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="8" 2023-09-06T18:20:08.940 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2023-09-06T18:20:08.940 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2023-09-06T18:20:08.940 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="8" 2023-09-06T18:20:08.940 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el8" 2023-09-06T18:20:08.940 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-06T18:20:08.941 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2023-09-06T18:20:08.941 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-06T18:20:08.941 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2023-09-06T18:20:08.941 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-06T18:20:08.941 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-06T18:20:08.942 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-06T18:20:08.942 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2023-09-06T18:20:08.979 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2023-09-06T18:20:09.001 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2023-09-06T18:20:09.002 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-09-06T18:20:09.062 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2023-09-06T18:20:09.062 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8" 2023-09-06T18:20:09.063 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2023-09-06T18:20:09.063 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2023-09-06T18:20:09.063 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8" 2023-09-06T18:20:09.063 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-09-06T18:20:09.064 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-06T18:20:09.064 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-09-06T18:20:09.064 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-06T18:20:09.064 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2023-09-06T18:20:09.064 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-06T18:20:09.065 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-06T18:20:09.065 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-06T18:20:09.065 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-09-06T18:20:09.103 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-06T18:20:09.115 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-06T18:20:09.125 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-06T18:20:09.126 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2023-09-06T18:20:09.128 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2023-09-06T18:20:09.148 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-06T18:20:09.158 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-06T18:20:09.158 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2023-09-06T18:20:09.189 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2023-09-06T18:20:09.216 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-06T18:20:09.295 INFO:teuthology.run_tasks:Running task kernel... 2023-09-06T18:20:09.313 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-09-06T18:20:09.314 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-09-06T18:20:09.314 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-06T18:20:09.315 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-06T18:20:09.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-06T18:20:09.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-06T18:20:09.337 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2023-09-06T18:20:09.337 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2023-09-06T18:20:09.397 INFO:teuthology.orchestra.run.smithi181.stdout:4.18.0-489.el8.x86_64 2023-09-06T18:20:09.398 INFO:teuthology.orchestra.run.smithi099.stdout:4.18.0-489.el8.x86_64 2023-09-06T18:20:09.398 INFO:teuthology.task.kernel:Running kernel on smithi181: 4.18.0-489.el8.x86_64 2023-09-06T18:20:09.399 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum install -y kernel 2023-09-06T18:20:09.399 INFO:teuthology.task.kernel:Running kernel on smithi099: 4.18.0-489.el8.x86_64 2023-09-06T18:20:09.399 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum install -y kernel 2023-09-06T18:20:19.147 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 8 - AppStream 4.7 MB/s | 33 MB 00:07 2023-09-06T18:20:19.156 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - AppStream 4.6 MB/s | 33 MB 00:07 2023-09-06T18:20:28.130 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 8 - BaseOS 16 MB/s | 47 MB 00:02 2023-09-06T18:20:28.539 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - BaseOS 14 MB/s | 47 MB 00:03 2023-09-06T18:20:35.529 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 8 - Extras 28 kB/s | 18 kB 00:00 2023-09-06T18:20:35.856 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - Extras 36 kB/s | 18 kB 00:00 2023-09-06T18:20:35.887 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 8 - Extras common packages 39 kB/s | 6.9 kB 00:00 2023-09-06T18:20:36.197 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-09-06T18:20:36.398 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - Extras common packages 19 kB/s | 6.9 kB 00:00 2023-09-06T18:20:36.699 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-09-06T18:20:37.248 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-09-06T18:20:38.425 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-09-06T18:20:40.989 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 511 kB/s | 24 kB 00:00 2023-09-06T18:20:42.192 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 418 kB/s | 24 kB 00:00 2023-09-06T18:20:44.405 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-09-06T18:20:44.406 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-09-06T18:20:44.660 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:20:44.661 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:20:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2023-09-06T18:20:44.662 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:20:44.662 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:20:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: kernel x86_64 4.18.0-513.el8 baseos 10 M 2023-09-06T18:20:44.662 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-09-06T18:20:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-core x86_64 4.18.0-513.el8 baseos 43 M 2023-09-06T18:20:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules x86_64 4.18.0-513.el8 baseos 35 M 2023-09-06T18:20:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:20:44.663 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:20:44.663 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:20:44.664 INFO:teuthology.orchestra.run.smithi099.stdout:Install 3 Packages 2023-09-06T18:20:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:20:44.664 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 88 M 2023-09-06T18:20:44.665 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 96 M 2023-09-06T18:20:44.665 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:20:45.666 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-09-06T18:20:45.666 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-09-06T18:20:45.906 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:20:45.907 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:20:45.907 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2023-09-06T18:20:45.907 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:20:45.907 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:20:45.908 INFO:teuthology.orchestra.run.smithi181.stdout: kernel x86_64 4.18.0-513.el8 baseos 10 M 2023-09-06T18:20:45.908 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-09-06T18:20:45.908 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-core x86_64 4.18.0-513.el8 baseos 43 M 2023-09-06T18:20:45.908 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-modules x86_64 4.18.0-513.el8 baseos 35 M 2023-09-06T18:20:45.908 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:20:45.909 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:20:45.909 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:20:45.909 INFO:teuthology.orchestra.run.smithi181.stdout:Install 3 Packages 2023-09-06T18:20:45.909 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:20:45.910 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 88 M 2023-09-06T18:20:45.910 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 96 M 2023-09-06T18:20:45.910 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:20:46.882 INFO:teuthology.orchestra.run.smithi099.stdout:(1/3): kernel-4.18.0-513.el8.x86_64.rpm 4.6 MB/s | 10 MB 00:02 2023-09-06T18:20:47.593 INFO:teuthology.orchestra.run.smithi181.stdout:(1/3): kernel-4.18.0-513.el8.x86_64.rpm 6.1 MB/s | 10 MB 00:01 2023-09-06T18:20:51.465 INFO:teuthology.orchestra.run.smithi099.stdout:(2/3): kernel-core-4.18.0-513.el8.x86_64.rpm 6.3 MB/s | 43 MB 00:06 2023-09-06T18:20:52.465 INFO:teuthology.orchestra.run.smithi099.stdout:(3/3): kernel-modules-4.18.0-513.el8.x86_64.rpm 4.6 MB/s | 35 MB 00:07 2023-09-06T18:20:52.466 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:20:52.466 INFO:teuthology.orchestra.run.smithi099.stdout:Total 11 MB/s | 88 MB 00:07 2023-09-06T18:20:53.138 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:20:53.599 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:20:53.599 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:20:54.811 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:20:54.826 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:20:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:20:58.295 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-core-4.18.0-513.el8.x86_64 1/3 2023-09-06T18:21:00.042 INFO:teuthology.orchestra.run.smithi181.stdout:(2/3): kernel-core-4.18.0-513.el8.x86_64.rpm 3.0 MB/s | 43 MB 00:14 2023-09-06T18:21:01.267 INFO:teuthology.orchestra.run.smithi181.stdout:(3/3): kernel-modules-4.18.0-513.el8.x86_64.rpm 2.3 MB/s | 35 MB 00:15 2023-09-06T18:21:01.267 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:21:01.268 INFO:teuthology.orchestra.run.smithi181.stdout:Total 5.7 MB/s | 88 MB 00:15 2023-09-06T18:21:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-core-4.18.0-513.el8.x86_64 1/3 2023-09-06T18:21:01.940 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:21:02.194 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-modules-4.18.0-513.el8.x86_64 2/3 2023-09-06T18:21:02.367 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:21:02.367 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:21:03.592 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:21:03.617 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:21:05.580 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-modules-4.18.0-513.el8.x86_64 2/3 2023-09-06T18:21:05.874 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-4.18.0-513.el8.x86_64 3/3 2023-09-06T18:21:06.420 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:21:06.973 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : kernel-core-4.18.0-513.el8.x86_64 1/3 2023-09-06T18:21:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-core-4.18.0-513.el8.x86_64 1/3 2023-09-06T18:21:11.106 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : kernel-modules-4.18.0-513.el8.x86_64 2/3 2023-09-06T18:21:14.493 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-modules-4.18.0-513.el8.x86_64 2/3 2023-09-06T18:21:14.794 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : kernel-4.18.0-513.el8.x86_64 3/3 2023-09-06T18:21:27.415 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-core-4.18.0-513.el8.x86_64 3/3 2023-09-06T18:21:27.438 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-modules-4.18.0-513.el8.x86_64 3/3 2023-09-06T18:21:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-4.18.0-513.el8.x86_64 3/3 2023-09-06T18:21:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-4.18.0-513.el8.x86_64 1/3 2023-09-06T18:21:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-core-4.18.0-513.el8.x86_64 2/3 2023-09-06T18:21:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-modules-4.18.0-513.el8.x86_64 3/3 2023-09-06T18:21:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:21:28.470 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:21:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-4.18.0-513.el8.x86_64 kernel-core-4.18.0-513.el8.x86_64 2023-09-06T18:21:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules-4.18.0-513.el8.x86_64 2023-09-06T18:21:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:21:28.471 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:21:28.613 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q kernel | sort -rV | head -n 1 2023-09-06T18:21:28.832 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-4.18.0-513.el8.x86_64 2023-09-06T18:21:28.832 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.el8.x86_64 2023-09-06T18:21:28.832 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-513.el8.x86_64 2023-09-06T18:21:28.833 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-06T18:21:28.833 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-06T18:21:28.833 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-09-06T18:21:28.834 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=None, version=distro) 2023-09-06T18:21:28.834 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q kernel | sort -rV | head -n 1 2023-09-06T18:21:29.041 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-4.18.0-513.el8.x86_64 2023-09-06T18:21:29.042 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.el8.x86_64 2023-09-06T18:21:29.042 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -qi grub2-tools 2023-09-06T18:21:29.075 INFO:teuthology.orchestra.run.smithi099.stdout:Name : grub2-tools 2023-09-06T18:21:29.076 INFO:teuthology.orchestra.run.smithi099.stdout:Epoch : 1 2023-09-06T18:21:29.076 INFO:teuthology.orchestra.run.smithi099.stdout:Version : 2.02 2023-09-06T18:21:29.076 INFO:teuthology.orchestra.run.smithi099.stdout:Release : 129.el8 2023-09-06T18:21:29.076 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: x86_64 2023-09-06T18:21:29.077 INFO:teuthology.orchestra.run.smithi099.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-09-06T18:21:29.077 INFO:teuthology.orchestra.run.smithi099.stdout:Group : System Environment/Base 2023-09-06T18:21:29.077 INFO:teuthology.orchestra.run.smithi099.stdout:Size : 9265586 2023-09-06T18:21:29.077 INFO:teuthology.orchestra.run.smithi099.stdout:License : GPLv3+ 2023-09-06T18:21:29.078 INFO:teuthology.orchestra.run.smithi099.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-09-06T18:21:29.078 INFO:teuthology.orchestra.run.smithi099.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-09-06T18:21:29.078 INFO:teuthology.orchestra.run.smithi099.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-09-06T18:21:29.078 INFO:teuthology.orchestra.run.smithi099.stdout:Build Host : kbuilder.bsys.centos.org 2023-09-06T18:21:29.078 INFO:teuthology.orchestra.run.smithi099.stdout:Relocations : (not relocatable) 2023-09-06T18:21:29.079 INFO:teuthology.orchestra.run.smithi099.stdout:Packager : CentOS BuildSystem 2023-09-06T18:21:29.079 INFO:teuthology.orchestra.run.smithi099.stdout:Vendor : CentOS 2023-09-06T18:21:29.079 INFO:teuthology.orchestra.run.smithi099.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-06T18:21:29.079 INFO:teuthology.orchestra.run.smithi099.stdout:Summary : Support tools for GRUB. 2023-09-06T18:21:29.079 INFO:teuthology.orchestra.run.smithi099.stdout:Description : 2023-09-06T18:21:29.079 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:21:29.080 INFO:teuthology.orchestra.run.smithi099.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-06T18:21:29.080 INFO:teuthology.orchestra.run.smithi099.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-06T18:21:29.080 INFO:teuthology.orchestra.run.smithi099.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-06T18:21:29.080 INFO:teuthology.orchestra.run.smithi099.stdout:hardware devices. 2023-09-06T18:21:29.081 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:21:29.081 INFO:teuthology.orchestra.run.smithi099.stdout:This subpackage provides tools for support of all platforms. 2023-09-06T18:21:29.082 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-06T18:21:29.082 INFO:teuthology.task.kernel:Updating grub on smithi099 to boot 4.18.0-513.el8.x86_64 2023-09-06T18:21:29.082 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-06T18:21:30.687 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2023-09-06T18:21:32.471 INFO:teuthology.orchestra.run.smithi099.stderr:done 2023-09-06T18:21:32.473 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2023-09-06T18:21:32.509 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.HwFX8Zyw5w 2023-09-06T18:21:32.510 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HwFX8Zyw5w 2023-09-06T18:21:32.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.HwFX8Zyw5w 2023-09-06T18:21:32.731 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.HwFX8Zyw5w is 6KB 2023-09-06T18:21:32.779 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.HwFX8Zyw5w 2023-09-06T18:21:32.798 DEBUG:teuthology.orchestra.run.smithi099:> sudo /bin/ls /boot/loader/entries || true 2023-09-06T18:21:32.885 INFO:teuthology.orchestra.run.smithi099.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-09-06T18:21:32.886 INFO:teuthology.orchestra.run.smithi099.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-09-06T18:21:32.886 INFO:teuthology.orchestra.run.smithi099.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-09-06T18:21:32.886 INFO:teuthology.orchestra.run.smithi099.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-513.el8.x86_64.conf 2023-09-06T18:21:32.887 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-513.el8.x86_64 2023-09-06T18:21:32.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2023-09-06T18:21:36.407 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-core-4.18.0-513.el8.x86_64 3/3 2023-09-06T18:21:36.431 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-modules-4.18.0-513.el8.x86_64 3/3 2023-09-06T18:21:36.771 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-4.18.0-513.el8.x86_64 3/3 2023-09-06T18:21:36.771 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-4.18.0-513.el8.x86_64 1/3 2023-09-06T18:21:36.772 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-core-4.18.0-513.el8.x86_64 2/3 2023-09-06T18:21:37.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-modules-4.18.0-513.el8.x86_64 3/3 2023-09-06T18:21:37.861 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:21:37.861 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:21:37.861 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-4.18.0-513.el8.x86_64 kernel-core-4.18.0-513.el8.x86_64 2023-09-06T18:21:37.862 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-modules-4.18.0-513.el8.x86_64 2023-09-06T18:21:37.862 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:21:37.862 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:21:38.006 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q kernel | sort -rV | head -n 1 2023-09-06T18:21:38.219 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-4.18.0-513.el8.x86_64 2023-09-06T18:21:38.219 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.el8.x86_64 2023-09-06T18:21:38.219 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-513.el8.x86_64 2023-09-06T18:21:38.220 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-06T18:21:38.220 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-06T18:21:38.220 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-09-06T18:21:38.220 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi181.front.sepia.ceph.com, path=None, version=distro) 2023-09-06T18:21:38.220 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q kernel | sort -rV | head -n 1 2023-09-06T18:21:38.429 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-4.18.0-513.el8.x86_64 2023-09-06T18:21:38.429 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.el8.x86_64 2023-09-06T18:21:38.429 DEBUG:teuthology.orchestra.run.smithi181:> sudo rpm -qi grub2-tools 2023-09-06T18:21:38.463 INFO:teuthology.orchestra.run.smithi181.stdout:Name : grub2-tools 2023-09-06T18:21:38.463 INFO:teuthology.orchestra.run.smithi181.stdout:Epoch : 1 2023-09-06T18:21:38.463 INFO:teuthology.orchestra.run.smithi181.stdout:Version : 2.02 2023-09-06T18:21:38.464 INFO:teuthology.orchestra.run.smithi181.stdout:Release : 129.el8 2023-09-06T18:21:38.464 INFO:teuthology.orchestra.run.smithi181.stdout:Architecture: x86_64 2023-09-06T18:21:38.464 INFO:teuthology.orchestra.run.smithi181.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-09-06T18:21:38.464 INFO:teuthology.orchestra.run.smithi181.stdout:Group : System Environment/Base 2023-09-06T18:21:38.464 INFO:teuthology.orchestra.run.smithi181.stdout:Size : 9265586 2023-09-06T18:21:38.465 INFO:teuthology.orchestra.run.smithi181.stdout:License : GPLv3+ 2023-09-06T18:21:38.465 INFO:teuthology.orchestra.run.smithi181.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-09-06T18:21:38.465 INFO:teuthology.orchestra.run.smithi181.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-09-06T18:21:38.465 INFO:teuthology.orchestra.run.smithi181.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-09-06T18:21:38.465 INFO:teuthology.orchestra.run.smithi181.stdout:Build Host : kbuilder.bsys.centos.org 2023-09-06T18:21:38.466 INFO:teuthology.orchestra.run.smithi181.stdout:Relocations : (not relocatable) 2023-09-06T18:21:38.466 INFO:teuthology.orchestra.run.smithi181.stdout:Packager : CentOS BuildSystem 2023-09-06T18:21:38.466 INFO:teuthology.orchestra.run.smithi181.stdout:Vendor : CentOS 2023-09-06T18:21:38.466 INFO:teuthology.orchestra.run.smithi181.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-06T18:21:38.466 INFO:teuthology.orchestra.run.smithi181.stdout:Summary : Support tools for GRUB. 2023-09-06T18:21:38.466 INFO:teuthology.orchestra.run.smithi181.stdout:Description : 2023-09-06T18:21:38.467 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:21:38.467 INFO:teuthology.orchestra.run.smithi181.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-06T18:21:38.467 INFO:teuthology.orchestra.run.smithi181.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-06T18:21:38.467 INFO:teuthology.orchestra.run.smithi181.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-06T18:21:38.467 INFO:teuthology.orchestra.run.smithi181.stdout:hardware devices. 2023-09-06T18:21:38.468 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:21:38.468 INFO:teuthology.orchestra.run.smithi181.stdout:This subpackage provides tools for support of all platforms. 2023-09-06T18:21:38.469 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-06T18:21:38.469 INFO:teuthology.task.kernel:Updating grub on smithi181 to boot 4.18.0-513.el8.x86_64 2023-09-06T18:21:38.469 DEBUG:teuthology.orchestra.run.smithi181:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-06T18:21:39.811 INFO:teuthology.orchestra.run.smithi181.stderr:Generating grub configuration file ... 2023-09-06T18:21:41.825 INFO:teuthology.orchestra.run.smithi181.stderr:done 2023-09-06T18:21:41.827 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2023-09-06T18:21:41.850 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.WlcLhBcjlX 2023-09-06T18:21:41.850 DEBUG:teuthology.orchestra.run.smithi181:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.WlcLhBcjlX 2023-09-06T18:21:41.924 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.WlcLhBcjlX 2023-09-06T18:21:42.061 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.WlcLhBcjlX is 6KB 2023-09-06T18:21:42.084 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.WlcLhBcjlX 2023-09-06T18:21:42.102 DEBUG:teuthology.orchestra.run.smithi181:> sudo /bin/ls /boot/loader/entries || true 2023-09-06T18:21:42.177 INFO:teuthology.orchestra.run.smithi181.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-09-06T18:21:42.178 INFO:teuthology.orchestra.run.smithi181.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-09-06T18:21:42.178 INFO:teuthology.orchestra.run.smithi181.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-09-06T18:21:42.178 INFO:teuthology.orchestra.run.smithi181.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-513.el8.x86_64.conf 2023-09-06T18:21:42.179 DEBUG:teuthology.orchestra.run.smithi181:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-513.el8.x86_64 2023-09-06T18:21:42.368 DEBUG:teuthology.orchestra.run.smithi181:> sudo shutdown -r now 2023-09-06T18:22:03.010 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-09-06T18:22:03.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-09-06T18:22:03.012 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-06T18:22:12.378 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-09-06T18:22:12.379 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-09-06T18:22:12.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-06T18:22:21.338 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-09-06T18:22:30.346 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-09-06T18:22:30.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-06T18:22:30.810 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-09-06T18:22:33.434 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-09-06T18:22:39.814 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-09-06T18:22:39.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-06T18:22:45.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-09-06T18:22:45.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-06T18:23:17.662 DEBUG:teuthology.orchestra.run.smithi099:> true 2023-09-06T18:23:18.753 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-09-06T18:23:18.753 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-513.el8.x86_64"... 2023-09-06T18:23:18.754 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2023-09-06T18:23:18.771 INFO:teuthology.orchestra.run.smithi099.stdout:4.18.0-513.el8.x86_64 2023-09-06T18:23:18.772 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.el8.x86_64 vs 4.18.0-513.el8.x86_64 2023-09-06T18:23:18.772 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-06T18:23:18.772 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-09-06T18:23:19.773 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-09-06T18:23:19.774 DEBUG:teuthology.orchestra.run.smithi099:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-06T18:23:19.849 INFO:teuthology.orchestra.run.smithi099.stdout:ttyS1 2023-09-06T18:23:19.865 DEBUG:teuthology.parallel:result is None 2023-09-06T18:23:39.848 DEBUG:teuthology.orchestra.remote:timed out 2023-09-06T18:23:51.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-09-06T18:23:51.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-06T18:23:52.231 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-09-06T18:23:52.533 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-09-06T18:23:52.534 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-513.el8.x86_64"... 2023-09-06T18:23:52.534 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2023-09-06T18:23:52.554 INFO:teuthology.orchestra.run.smithi181.stdout:4.18.0-513.el8.x86_64 2023-09-06T18:23:52.554 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.el8.x86_64 vs 4.18.0-513.el8.x86_64 2023-09-06T18:23:52.554 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-06T18:23:52.555 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-09-06T18:23:53.556 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-09-06T18:23:53.556 DEBUG:teuthology.orchestra.run.smithi181:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-06T18:23:53.657 INFO:teuthology.orchestra.run.smithi181.stdout:ttyS1 2023-09-06T18:23:53.671 DEBUG:teuthology.parallel:result is None 2023-09-06T18:23:53.672 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-06T18:23:53.680 INFO:teuthology.task.internal:Creating test directory... 2023-09-06T18:23:53.681 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-06T18:23:53.685 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-06T18:23:53.706 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-06T18:23:53.713 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-06T18:23:53.722 INFO:teuthology.task.internal:Creating archive directory... 2023-09-06T18:23:53.724 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-06T18:23:53.748 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-06T18:23:53.785 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-06T18:23:53.792 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-06T18:23:53.793 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-06T18:23:53.819 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-06T18:23:53.852 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-06T18:23:53.855 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-06T18:23:53.864 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-06T18:23:53.866 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-06T18:23:53.869 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-06T18:23:53.883 INFO:teuthology.task.internal:Configuring sudo... 2023-09-06T18:23:53.885 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-06T18:23:53.918 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-06T18:23:53.948 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-06T18:23:53.958 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-06T18:23:53.958 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-06T18:23:53.989 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-06T18:23:54.012 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-06T18:23:54.109 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-06T18:23:54.158 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-06T18:23:54.258 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-06T18:23:54.291 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:23:54.291 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-06T18:23:54.358 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-06T18:23:54.418 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-06T18:23:54.457 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-06T18:23:54.554 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-06T18:23:54.588 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:23:54.588 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-06T18:23:54.656 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2023-09-06T18:23:54.659 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2023-09-06T18:23:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-06T18:23:54.773 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-06T18:23:55.231 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-06T18:23:55.240 INFO:teuthology.task.internal:Starting timer... 2023-09-06T18:23:55.240 INFO:teuthology.run_tasks:Running task pcp... 2023-09-06T18:23:55.257 INFO:teuthology.run_tasks:Running task selinux... 2023-09-06T18:23:55.267 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-09-06T18:23:55.268 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2023-09-06T18:23:55.362 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2023-09-06T18:23:55.363 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2023-09-06T18:23:55.447 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2023-09-06T18:23:55.449 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-06T18:23:55.449 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2023-09-06T18:23:55.476 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2023-09-06T18:23:55.477 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2023-09-06T18:23:55.521 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2023-09-06T18:23:55.522 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive'} 2023-09-06T18:23:55.522 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-06T18:23:55.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-06T18:23:55.554 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-06T18:23:55.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-06T18:23:55.583 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-06T18:23:55.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2023-09-06T18:23:55.623 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2023-09-06T18:23:55.652 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-06T18:23:55.664 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-09-06T18:23:55.665 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-06T18:23:55.687 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-06T18:23:55.689 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi099.front.sepia.ceph.com,smithi181.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-06T18:33:00.605 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com')] 2023-09-06T18:33:00.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-09-06T18:33:00.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-06T18:33:00.739 DEBUG:teuthology.orchestra.run.smithi099:> true 2023-09-06T18:33:00.812 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-09-06T18:33:00.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-09-06T18:33:00.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-06T18:33:00.954 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-09-06T18:33:01.026 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-09-06T18:33:01.027 INFO:teuthology.run_tasks:Running task clock... 2023-09-06T18:33:01.045 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-06T18:33:01.047 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-06T18:33:01.048 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-06T18:33:01.052 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-06T18:33:01.053 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-06T18:33:01.083 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-06T18:33:01.098 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-06T18:33:01.102 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-06T18:33:01.117 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-06T18:33:01.168 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2023-09-06T18:33:01.180 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2023-09-06T18:33:01.181 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2023-09-06T18:33:01.192 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2023-09-06T18:33:01.193 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-06T18:33:01.209 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-06T18:33:01.210 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-06T18:33:01.219 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-06T18:33:01.276 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2023-09-06T18:33:01.279 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-06T18:33:01.280 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2023-09-06T18:33:01.280 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-06T18:33:01.280 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-06T18:33:01.280 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-06T18:33:01.280 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-06T18:33:01.291 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2023-09-06T18:33:01.295 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-06T18:33:01.295 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-09-06T18:33:01.296 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-06T18:33:01.296 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-06T18:33:01.296 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-06T18:33:01.296 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-06T18:33:01.297 INFO:teuthology.run_tasks:Running task install... 2023-09-06T18:33:01.306 DEBUG:teuthology.task.install:project ceph 2023-09-06T18:33:01.307 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5535a92bd618b7e580a2fce32e9dfab6620d1086'}} 2023-09-06T18:33:01.307 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '5535a92bd618b7e580a2fce32e9dfab6620d1086'} 2023-09-06T18:33:01.307 INFO:teuthology.task.install:Using flavor: default 2023-09-06T18:33:01.314 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-06T18:33:01.314 INFO:teuthology.task.install:extra packages: [] 2023-09-06T18:33:01.315 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5535a92bd618b7e580a2fce32e9dfab6620d1086', 'tag': None, 'wait_for_package': False} 2023-09-06T18:33:01.315 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5535a92bd618b7e580a2fce32e9dfab6620d1086 2023-09-06T18:33:01.318 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5535a92bd618b7e580a2fce32e9dfab6620d1086', 'tag': None, 'wait_for_package': False} 2023-09-06T18:33:01.318 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5535a92bd618b7e580a2fce32e9dfab6620d1086 2023-09-06T18:33:01.504 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-eric-testing-1/5535a92bd618b7e580a2fce32e9dfab6620d1086/centos/8/flavors/default/ 2023-09-06T18:33:01.504 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5801.g5535a92b 2023-09-06T18:33:01.623 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-eric-testing-1/5535a92bd618b7e580a2fce32e9dfab6620d1086/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-eric-testing-1/5535a92bd618b7e580a2fce32e9dfab6620d1086/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-eric-testing-1/5535a92bd618b7e580a2fce32e9dfab6620d1086/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-06T18:33:01.623 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:33:01.624 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-06T18:33:01.652 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-09-06T18:33:01.652 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5535a92bd618b7e580a2fce32e9dfab6620d1086/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-06T18:33:01.709 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-eric-testing-1/5535a92bd618b7e580a2fce32e9dfab6620d1086/centos/8/flavors/default/ 2023-09-06T18:33:01.709 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5801.g5535a92b 2023-09-06T18:33:01.721 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-06T18:33:01.800 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-06T18:33:01.828 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-eric-testing-1/5535a92bd618b7e580a2fce32e9dfab6620d1086/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-eric-testing-1/5535a92bd618b7e580a2fce32e9dfab6620d1086/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-eric-testing-1/5535a92bd618b7e580a2fce32e9dfab6620d1086/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-06T18:33:01.829 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:33:01.829 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-06T18:33:01.859 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-09-06T18:33:01.860 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5535a92bd618b7e580a2fce32e9dfab6620d1086/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-06T18:33:01.866 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2023-09-06T18:33:01.868 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2023-09-06T18:33:01.932 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-06T18:33:02.010 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-06T18:33:02.078 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2023-09-06T18:33:02.079 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2023-09-06T18:33:02.140 INFO:teuthology.orchestra.run.smithi181.stdout:76 files removed 2023-09-06T18:33:02.167 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw 2023-09-06T18:33:02.347 INFO:teuthology.orchestra.run.smithi099.stdout:76 files removed 2023-09-06T18:33:02.378 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw 2023-09-06T18:33:03.208 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 129 kB/s | 81 kB 00:00 2023-09-06T18:33:03.423 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 127 kB/s | 81 kB 00:00 2023-09-06T18:33:03.658 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 23 kB/s | 9.4 kB 00:00 2023-09-06T18:33:03.698 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 40 kB/s | 9.4 kB 00:00 2023-09-06T18:33:03.926 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 6.2 kB/s | 1.5 kB 00:00 2023-09-06T18:33:04.216 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 3.0 kB/s | 1.5 kB 00:00 2023-09-06T18:33:14.623 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 8 - AppStream 3.2 MB/s | 33 MB 00:10 2023-09-06T18:33:14.758 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - AppStream 3.1 MB/s | 33 MB 00:10 2023-09-06T18:33:24.331 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 8 - BaseOS 13 MB/s | 47 MB 00:03 2023-09-06T18:33:24.351 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - BaseOS 14 MB/s | 47 MB 00:03 2023-09-06T18:33:31.559 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - Extras 36 kB/s | 18 kB 00:00 2023-09-06T18:33:31.673 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 8 - Extras 28 kB/s | 18 kB 00:00 2023-09-06T18:33:32.076 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - Extras common packages 20 kB/s | 6.9 kB 00:00 2023-09-06T18:33:32.273 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 8 - Extras common packages 16 kB/s | 6.9 kB 00:00 2023-09-06T18:33:33.386 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - PowerTools 5.6 MB/s | 6.5 MB 00:01 2023-09-06T18:33:33.449 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 8 - PowerTools 6.3 MB/s | 6.5 MB 00:01 2023-09-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2023-09-06T18:33:34.892 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-09-06T18:33:38.145 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 5.1 MB/s | 16 MB 00:03 2023-09-06T18:33:42.042 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 392 kB/s | 24 kB 00:00 2023-09-06T18:33:46.439 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:33:46.441 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:33:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-06T18:33:46.442 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:33:46.442 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:33:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 14 M 2023-09-06T18:33:46.442 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2023-09-06T18:33:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 3.7 M 2023-09-06T18:33:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 3.6 M 2023-09-06T18:33:46.443 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-09-06T18:33:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 5.4 M 2023-09-06T18:33:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 22 M 2023-09-06T18:33:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 24 k 2023-09-06T18:33:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-06T18:33:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 783 k 2023-09-06T18:33:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-06T18:33:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-09-06T18:33:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 474 k 2023-09-06T18:33:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-09-06T18:33:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 6.8 M 2023-09-06T18:33:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-06T18:33:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-09-06T18:33:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 45 k 2023-09-06T18:33:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 125 k 2023-09-06T18:33:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 197 k 2023-09-06T18:33:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-09-06T18:33:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 371 k 2023-09-06T18:33:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 364 k 2023-09-06T18:33:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 107 k 2023-09-06T18:33:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-06T18:33:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:33:46.447 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:33:46.447 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:33:46.447 INFO:teuthology.orchestra.run.smithi099.stdout:Install 21 Packages 2023-09-06T18:33:46.448 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 2 Packages 2023-09-06T18:33:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:33:46.448 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 61 M 2023-09-06T18:33:46.449 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:33:46.937 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 16 MB 00:11 2023-09-06T18:33:47.801 INFO:teuthology.orchestra.run.smithi099.stdout:(1/23): ceph-base-18.0.0-5801.g5535a92b.el8.x86 8.2 MB/s | 5.4 MB 00:00 2023-09-06T18:33:47.851 INFO:teuthology.orchestra.run.smithi099.stdout:(2/23): ceph-selinux-18.0.0-5801.g5535a92b.el8. 486 kB/s | 24 kB 00:00 2023-09-06T18:33:47.985 INFO:teuthology.orchestra.run.smithi099.stdout:(3/23): libcephfs2-18.0.0-5801.g5535a92b.el8.x8 5.7 MB/s | 783 kB 00:00 2023-09-06T18:33:48.108 INFO:teuthology.orchestra.run.smithi099.stdout:(4/23): libradosstriper1-18.0.0-5801.g5535a92b. 3.8 MB/s | 474 kB 00:00 2023-09-06T18:33:48.252 INFO:teuthology.orchestra.run.smithi099.stdout:(5/23): ceph-radosgw-18.0.0-5801.g5535a92b.el8. 12 MB/s | 14 MB 00:01 2023-09-06T18:33:48.302 INFO:teuthology.orchestra.run.smithi099.stdout:(6/23): python3-ceph-argparse-18.0.0-5801.g5535 902 kB/s | 45 kB 00:00 2023-09-06T18:33:48.360 INFO:teuthology.orchestra.run.smithi099.stdout:(7/23): python3-ceph-common-18.0.0-5801.g5535a9 2.1 MB/s | 125 kB 00:00 2023-09-06T18:33:48.452 INFO:teuthology.orchestra.run.smithi099.stdout:(8/23): python3-cephfs-18.0.0-5801.g5535a92b.el 2.1 MB/s | 197 kB 00:00 2023-09-06T18:33:48.536 INFO:teuthology.orchestra.run.smithi099.stdout:(9/23): python3-rados-18.0.0-5801.g5535a92b.el8 4.3 MB/s | 371 kB 00:00 2023-09-06T18:33:48.628 INFO:teuthology.orchestra.run.smithi099.stdout:(10/23): librgw2-18.0.0-5801.g5535a92b.el8.x86_ 13 MB/s | 6.8 MB 00:00 2023-09-06T18:33:48.662 INFO:teuthology.orchestra.run.smithi099.stdout:(11/23): python3-rbd-18.0.0-5801.g5535a92b.el8. 2.8 MB/s | 364 kB 00:00 2023-09-06T18:33:48.678 INFO:teuthology.orchestra.run.smithi099.stdout:(12/23): python3-rgw-18.0.0-5801.g5535a92b.el8. 2.1 MB/s | 107 kB 00:00 2023-09-06T18:33:49.946 INFO:teuthology.orchestra.run.smithi099.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 205 kB/s | 259 kB 00:01 2023-09-06T18:33:50.262 INFO:teuthology.orchestra.run.smithi099.stdout:(14/23): ceph-common-18.0.0-5801.g5535a92b.el8. 7.1 MB/s | 22 MB 00:03 2023-09-06T18:33:50.288 INFO:teuthology.orchestra.run.smithi099.stdout:(15/23): librdkafka-0.11.4-3.el8.x86_64.rpm 218 kB/s | 354 kB 00:01 2023-09-06T18:33:50.429 INFO:teuthology.orchestra.run.smithi099.stdout:(16/23): librabbitmq-0.9.0-4.el8.x86_64.rpm 282 kB/s | 47 kB 00:00 2023-09-06T18:33:50.547 INFO:teuthology.orchestra.run.smithi099.stdout:(17/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-09-06T18:33:50.605 INFO:teuthology.orchestra.run.smithi099.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 335 kB/s | 59 kB 00:00 2023-09-06T18:33:50.630 INFO:teuthology.orchestra.run.smithi099.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 900 kB/s | 75 kB 00:00 2023-09-06T18:33:50.647 INFO:teuthology.orchestra.run.smithi099.stdout:(20/23): python3-prettytable-0.7.2-14.el8.noarc 63 kB/s | 44 kB 00:00 2023-09-06T18:33:50.720 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 422 kB/s | 24 kB 00:00 2023-09-06T18:33:50.856 INFO:teuthology.orchestra.run.smithi099.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 6.9 MB/s | 1.7 MB 00:00 2023-09-06T18:33:50.965 INFO:teuthology.orchestra.run.smithi099.stdout:(22/23): librados2-18.0.0-5801.g5535a92b.el8.x8 11 MB/s | 3.7 MB 00:00 2023-09-06T18:33:51.131 INFO:teuthology.orchestra.run.smithi099.stdout:(23/23): librbd1-18.0.0-5801.g5535a92b.el8.x86_ 7.5 MB/s | 3.6 MB 00:00 2023-09-06T18:33:51.134 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:33:51.134 INFO:teuthology.orchestra.run.smithi099.stdout:Total 13 MB/s | 61 MB 00:04 2023-09-06T18:33:51.177 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:33:51.223 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:33:51.223 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:33:51.667 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:33:51.671 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:33:53.055 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:33:53.239 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-09-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2023-09-06T18:33:53.672 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-09-06T18:33:54.096 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-09-06T18:33:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:18.0.0-5801.g5535a92b.el8.x86_64 4/25 2023-09-06T18:33:54.448 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:18.0.0-5801.g5535a92b.el8.x86_64 4/25 2023-09-06T18:33:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:18.0.0-5801.g5535a92b.el8.x86_64 5/25 2023-09-06T18:33:54.897 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:18.0.0-5801.g5535a92b.el8.x86_64 6/25 2023-09-06T18:33:55.107 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:33:55.109 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:33:55.110 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-09-06T18:33:55.110 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:33:55.110 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:33:55.110 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 14 M 2023-09-06T18:33:55.111 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2023-09-06T18:33:55.111 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 3.7 M 2023-09-06T18:33:55.111 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 3.6 M 2023-09-06T18:33:55.111 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-09-06T18:33:55.112 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 5.4 M 2023-09-06T18:33:55.112 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 22 M 2023-09-06T18:33:55.112 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 24 k 2023-09-06T18:33:55.112 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-06T18:33:55.112 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 783 k 2023-09-06T18:33:55.113 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-06T18:33:55.113 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-09-06T18:33:55.113 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 474 k 2023-09-06T18:33:55.113 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-09-06T18:33:55.114 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 6.8 M 2023-09-06T18:33:55.114 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-06T18:33:55.114 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-09-06T18:33:55.114 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 45 k 2023-09-06T18:33:55.115 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 125 k 2023-09-06T18:33:55.115 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 197 k 2023-09-06T18:33:55.115 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-09-06T18:33:55.115 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 371 k 2023-09-06T18:33:55.115 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 364 k 2023-09-06T18:33:55.116 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 107 k 2023-09-06T18:33:55.116 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-06T18:33:55.116 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:33:55.116 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:33:55.117 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:33:55.117 INFO:teuthology.orchestra.run.smithi181.stdout:Install 21 Packages 2023-09-06T18:33:55.117 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 2 Packages 2023-09-06T18:33:55.118 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:33:55.119 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 61 M 2023-09-06T18:33:55.119 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:33:55.293 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:18.0.0-5801.g5535a92b.el8.x86_64 6/25 2023-09-06T18:33:55.505 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:18.0.0-5801.g5535a92b.el8.x86_64 7/25 2023-09-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:18.0.0-5801.g5535a92b.el8.x86_64 7/25 2023-09-06T18:33:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 8/25 2023-09-06T18:33:55.955 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-09-06T18:33:56.016 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-09-06T18:33:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:18.0.0-5801.g5535a92b.el8. 10/25 2023-09-06T18:33:56.433 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:18.0.0-5801.g5535a92b.el8.x86_64 11/25 2023-09-06T18:33:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:18.0.0-5801.g5535a92b.el8.x86_64 12/25 2023-09-06T18:33:56.843 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:18.0.0-5801.g5535a92b.el8.x86_6 13/25 2023-09-06T18:33:56.922 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5801.g5535a92b.el8.x86_6 13/25 2023-09-06T18:33:57.018 INFO:teuthology.orchestra.run.smithi181.stdout:(1/23): ceph-radosgw-18.0.0-5801.g5535a92b.el8. 9.9 MB/s | 14 MB 00:01 2023-09-06T18:33:57.063 INFO:teuthology.orchestra.run.smithi181.stdout:(2/23): ceph-selinux-18.0.0-5801.g5535a92b.el8. 548 kB/s | 24 kB 00:00 2023-09-06T18:33:57.133 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-09-06T18:33:57.455 INFO:teuthology.orchestra.run.smithi181.stdout:(3/23): ceph-common-18.0.0-5801.g5535a92b.el8.x 12 MB/s | 22 MB 00:01 2023-09-06T18:33:57.497 INFO:teuthology.orchestra.run.smithi181.stdout:(4/23): libcephfs2-18.0.0-5801.g5535a92b.el8.x8 1.8 MB/s | 783 kB 00:00 2023-09-06T18:33:57.522 INFO:teuthology.orchestra.run.smithi181.stdout:(5/23): libradosstriper1-18.0.0-5801.g5535a92b. 7.0 MB/s | 474 kB 00:00 2023-09-06T18:33:57.597 INFO:teuthology.orchestra.run.smithi181.stdout:(6/23): ceph-base-18.0.0-5801.g5535a92b.el8.x86 2.8 MB/s | 5.4 MB 00:01 2023-09-06T18:33:57.623 INFO:teuthology.orchestra.run.smithi181.stdout:(7/23): python3-ceph-argparse-18.0.0-5801.g5535 445 kB/s | 45 kB 00:00 2023-09-06T18:33:57.673 INFO:teuthology.orchestra.run.smithi181.stdout:(8/23): python3-ceph-common-18.0.0-5801.g5535a9 1.6 MB/s | 125 kB 00:00 2023-09-06T18:33:57.698 INFO:teuthology.orchestra.run.smithi181.stdout:(9/23): python3-cephfs-18.0.0-5801.g5535a92b.el 2.6 MB/s | 197 kB 00:00 2023-09-06T18:33:57.757 INFO:teuthology.orchestra.run.smithi181.stdout:(10/23): python3-rados-18.0.0-5801.g5535a92b.el 4.3 MB/s | 371 kB 00:00 2023-09-06T18:33:57.816 INFO:teuthology.orchestra.run.smithi181.stdout:(11/23): python3-rbd-18.0.0-5801.g5535a92b.el8. 3.0 MB/s | 364 kB 00:00 2023-09-06T18:33:57.832 INFO:teuthology.orchestra.run.smithi181.stdout:(12/23): python3-rgw-18.0.0-5801.g5535a92b.el8. 1.4 MB/s | 107 kB 00:00 2023-09-06T18:33:57.925 INFO:teuthology.orchestra.run.smithi181.stdout:(13/23): librgw2-18.0.0-5801.g5535a92b.el8.x86_ 16 MB/s | 6.8 MB 00:00 2023-09-06T18:33:57.957 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-09-06T18:33:58.088 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:18.0.0-5801.g5535a92b.el8.x86_64 16/25 2023-09-06T18:33:58.150 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:18.0.0-5801.g5535a92b.el8.x86_64 16/25 2023-09-06T18:33:58.346 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:18.0.0-5801.g5535a92b.el8.x86_64 17/25 2023-09-06T18:33:58.450 INFO:teuthology.orchestra.run.smithi181.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 84 kB/s | 44 kB 00:00 2023-09-06T18:33:58.531 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-09-06T18:33:58.730 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:18.0.0-5801.g5535a92b.el8.x8 19/25 2023-09-06T18:33:58.986 INFO:teuthology.orchestra.run.smithi181.stdout:(15/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 226 kB/s | 259 kB 00:01 2023-09-06T18:33:59.475 INFO:teuthology.orchestra.run.smithi181.stdout:(16/23): librdkafka-0.11.4-3.el8.x86_64.rpm 213 kB/s | 354 kB 00:01 2023-09-06T18:33:59.526 INFO:teuthology.orchestra.run.smithi181.stdout:(17/23): gperftools-libs-2.7-9.el8.x86_64.rpm 565 kB/s | 306 kB 00:00 2023-09-06T18:33:59.660 INFO:teuthology.orchestra.run.smithi181.stdout:(18/23): libunwind-1.3.1-3.el8.x86_64.rpm 562 kB/s | 75 kB 00:00 2023-09-06T18:33:59.843 INFO:teuthology.orchestra.run.smithi181.stdout:(19/23): liboath-2.6.2-3.el8.x86_64.rpm 160 kB/s | 59 kB 00:00 2023-09-06T18:33:59.868 INFO:teuthology.orchestra.run.smithi181.stdout:(20/23): librabbitmq-0.9.0-4.el8.x86_64.rpm 33 kB/s | 47 kB 00:01 2023-09-06T18:34:00.044 INFO:teuthology.orchestra.run.smithi181.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 4.5 MB/s | 1.7 MB 00:00 2023-09-06T18:34:00.220 INFO:teuthology.orchestra.run.smithi181.stdout:(22/23): librbd1-18.0.0-5801.g5535a92b.el8.x86_ 10 MB/s | 3.6 MB 00:00 2023-09-06T18:34:00.552 INFO:teuthology.orchestra.run.smithi181.stdout:(23/23): librados2-18.0.0-5801.g5535a92b.el8.x8 5.2 MB/s | 3.7 MB 00:00 2023-09-06T18:34:00.555 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:34:00.555 INFO:teuthology.orchestra.run.smithi181.stdout:Total 11 MB/s | 61 MB 00:05 2023-09-06T18:34:00.598 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:34:00.644 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:34:00.645 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:34:00.986 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:34:00.989 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:34:01.356 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.0.0-5801.g5535a92b.el8.x86_64 20/25 2023-09-06T18:34:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:18.0.0-5801.g5535a92b.el8.x86_64 20/25 2023-09-06T18:34:02.176 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.0.0-5801.g5535a92b.el8.x86_64 20/25 2023-09-06T18:34:02.554 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:34:02.685 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:18.0.0-5801.g5535a92b.el8.x86_64 21/25 2023-09-06T18:34:02.738 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-09-06T18:34:02.795 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:18.0.0-5801.g5535a92b.el8.x86_64 21/25 2023-09-06T18:34:02.957 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2023-09-06T18:34:02.987 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:18.0.0-5801.g5535a92b.el8.x86_64 22/25 2023-09-06T18:34:03.128 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-09-06T18:34:03.530 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-09-06T18:34:03.761 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:18.0.0-5801.g5535a92b.el8.x86_64 4/25 2023-09-06T18:34:03.863 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:18.0.0-5801.g5535a92b.el8.x86_64 4/25 2023-09-06T18:34:04.151 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:18.0.0-5801.g5535a92b.el8.x86_64 5/25 2023-09-06T18:34:04.403 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:18.0.0-5801.g5535a92b.el8.x86_64 6/25 2023-09-06T18:34:04.797 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:18.0.0-5801.g5535a92b.el8.x86_64 6/25 2023-09-06T18:34:04.961 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:18.0.0-5801.g5535a92b.el8.x86_64 7/25 2023-09-06T18:34:05.017 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:18.0.0-5801.g5535a92b.el8.x86_64 7/25 2023-09-06T18:34:05.242 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 8/25 2023-09-06T18:34:05.436 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-09-06T18:34:05.504 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-09-06T18:34:05.655 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:18.0.0-5801.g5535a92b.el8. 10/25 2023-09-06T18:34:05.880 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:18.0.0-5801.g5535a92b.el8.x86_64 11/25 2023-09-06T18:34:06.086 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:18.0.0-5801.g5535a92b.el8.x86_64 12/25 2023-09-06T18:34:06.261 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:18.0.0-5801.g5535a92b.el8.x86_6 13/25 2023-09-06T18:34:06.334 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5801.g5535a92b.el8.x86_6 13/25 2023-09-06T18:34:06.587 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-09-06T18:34:07.471 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-09-06T18:34:07.627 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:18.0.0-5801.g5535a92b.el8.x86_64 16/25 2023-09-06T18:34:07.698 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:18.0.0-5801.g5535a92b.el8.x86_64 16/25 2023-09-06T18:34:07.869 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:18.0.0-5801.g5535a92b.el8.x86_64 17/25 2023-09-06T18:34:08.087 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-09-06T18:34:08.244 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:18.0.0-5801.g5535a92b.el8.x8 19/25 2023-09-06T18:34:10.845 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.0.0-5801.g5535a92b.el8.x86_64 20/25 2023-09-06T18:34:11.046 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:18.0.0-5801.g5535a92b.el8.x86_64 20/25 2023-09-06T18:34:11.647 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.0.0-5801.g5535a92b.el8.x86_64 20/25 2023-09-06T18:34:11.953 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:18.0.0-5801.g5535a92b.el8.x86_64 21/25 2023-09-06T18:34:12.054 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:18.0.0-5801.g5535a92b.el8.x86_64 21/25 2023-09-06T18:34:12.194 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:18.0.0-5801.g5535a92b.el8.x86_64 22/25 2023-09-06T18:34:21.682 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5801.g5535a92b.el8.x86_64 22/25 2023-09-06T18:34:21.682 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2023-09-06T18:34:21.682 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2023-09-06T18:34:21.683 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2023-09-06T18:34:21.683 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /run 2023-09-06T18:34:21.683 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2023-09-06T18:34:21.683 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2023-09-06T18:34:21.683 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2023-09-06T18:34:21.684 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2023-09-06T18:34:21.684 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2023-09-06T18:34:21.684 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:34:23.122 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:18.0.0-5801.g5535a92b.el8.x86_64 23/25 2023-09-06T18:34:23.142 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5801.g5535a92b.el8.x86_64 23/25 2023-09-06T18:34:23.143 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-09-06T18:34:23.401 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-09-06T18:34:23.402 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-09-06T18:34:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-09-06T18:34:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:18.0.0-5801.g5535a92b.el8.x86_64 1/25 2023-09-06T18:34:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:18.0.0-5801.g5535a92b.el8.x86_64 2/25 2023-09-06T18:34:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:18.0.0-5801.g5535a92b.el8.x86_64 3/25 2023-09-06T18:34:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:18.0.0-5801.g5535a92b.el8.x86_64 4/25 2023-09-06T18:34:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:18.0.0-5801.g5535a92b.el8.x86_64 5/25 2023-09-06T18:34:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:18.0.0-5801.g5535a92b.el8.x86_6 6/25 2023-09-06T18:34:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:18.0.0-5801.g5535a92b.el8.x86_64 7/25 2023-09-06T18:34:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5801.g5535a92b.el8. 8/25 2023-09-06T18:34:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:18.0.0-5801.g5535a92b.el8.x8 9/25 2023-09-06T18:34:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:18.0.0-5801.g5535a92b.el8.x86_64 10/25 2023-09-06T18:34:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:18.0.0-5801.g5535a92b.el8.x86_64 11/25 2023-09-06T18:34:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:18.0.0-5801.g5535a92b.el8.x86_64 12/25 2023-09-06T18:34:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:18.0.0-5801.g5535a92b.el8.x86_64 13/25 2023-09-06T18:34:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-09-06T18:34:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-09-06T18:34:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-09-06T18:34:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 17/25 2023-09-06T18:34:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-09-06T18:34:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-09-06T18:34:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-09-06T18:34:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-09-06T18:34:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:18.0.0-5801.g5535a92b.el8.x86_64 22/25 2023-09-06T18:34:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-09-06T18:34:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:18.0.0-5801.g5535a92b.el8.x86_64 24/25 2023-09-06T18:34:25.292 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-09-06T18:34:25.293 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:34:25.293 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2023-09-06T18:34:25.293 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.293 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.294 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:34:25.294 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.294 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.295 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.295 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.295 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-06T18:34:25.295 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.295 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-06T18:34:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-09-06T18:34:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-06T18:34:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-06T18:34:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-06T18:34:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-06T18:34:25.298 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.298 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.298 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:25.298 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-06T18:34:25.298 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:34:25.299 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:34:25.615 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-test 2023-09-06T18:34:26.458 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:45 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:34:28.013 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:34:28.014 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:34:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-06T18:34:28.015 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:34:28.015 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:34:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 48 M 2023-09-06T18:34:28.016 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-09-06T18:34:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-09-06T18:34:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 139 k 2023-09-06T18:34:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-09-06T18:34:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-09-06T18:34:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-09-06T18:34:28.018 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:34:28.018 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:34:28.018 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:34:28.019 INFO:teuthology.orchestra.run.smithi099.stdout:Install 6 Packages 2023-09-06T18:34:28.019 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:34:28.020 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 49 M 2023-09-06T18:34:28.020 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 218 M 2023-09-06T18:34:28.020 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:34:28.389 INFO:teuthology.orchestra.run.smithi099.stdout:(1/6): libcephsqlite-18.0.0-5801.g5535a92b.el8. 768 kB/s | 139 kB 00:00 2023-09-06T18:34:29.290 INFO:teuthology.orchestra.run.smithi099.stdout:(2/6): jq-1.6-7.el8.x86_64.rpm 187 kB/s | 202 kB 00:01 2023-09-06T18:34:29.348 INFO:teuthology.orchestra.run.smithi099.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 195 kB/s | 187 kB 00:00 2023-09-06T18:34:29.982 INFO:teuthology.orchestra.run.smithi099.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 111 kB/s | 71 kB 00:00 2023-09-06T18:34:30.582 INFO:teuthology.orchestra.run.smithi099.stdout:(5/6): socat-1.7.4.1-1.el8.x86_64.rpm 250 kB/s | 323 kB 00:01 2023-09-06T18:34:30.661 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5801.g5535a92b.el8.x86_64 22/25 2023-09-06T18:34:30.661 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2023-09-06T18:34:30.662 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2023-09-06T18:34:30.662 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2023-09-06T18:34:30.662 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /run 2023-09-06T18:34:30.662 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2023-09-06T18:34:30.663 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2023-09-06T18:34:30.663 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2023-09-06T18:34:30.663 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2023-09-06T18:34:30.663 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2023-09-06T18:34:30.663 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:34:31.432 INFO:teuthology.orchestra.run.smithi099.stdout:(6/6): ceph-test-18.0.0-5801.g5535a92b.el8.x86_ 15 MB/s | 48 MB 00:03 2023-09-06T18:34:31.434 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:34:31.434 INFO:teuthology.orchestra.run.smithi099.stdout:Total 14 MB/s | 49 MB 00:03 2023-09-06T18:34:31.479 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:34:31.500 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:34:31.501 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:34:31.836 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:34:31.839 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:34:32.092 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:18.0.0-5801.g5535a92b.el8.x86_64 23/25 2023-09-06T18:34:32.113 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5801.g5535a92b.el8.x86_64 23/25 2023-09-06T18:34:32.113 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-09-06T18:34:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:34:32.342 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-09-06T18:34:32.343 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-09-06T18:34:32.557 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-09-06T18:34:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-09-06T18:34:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-09-06T18:34:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-09-06T18:34:33.064 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-09-06T18:34:33.064 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:18.0.0-5801.g5535a92b.el8.x86_64 1/25 2023-09-06T18:34:33.065 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:18.0.0-5801.g5535a92b.el8.x86_64 2/25 2023-09-06T18:34:33.065 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:18.0.0-5801.g5535a92b.el8.x86_64 3/25 2023-09-06T18:34:33.065 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:18.0.0-5801.g5535a92b.el8.x86_64 4/25 2023-09-06T18:34:33.068 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:18.0.0-5801.g5535a92b.el8.x86_64 5/25 2023-09-06T18:34:33.068 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:18.0.0-5801.g5535a92b.el8.x86_6 6/25 2023-09-06T18:34:33.068 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:18.0.0-5801.g5535a92b.el8.x86_64 7/25 2023-09-06T18:34:33.069 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5801.g5535a92b.el8. 8/25 2023-09-06T18:34:33.069 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:18.0.0-5801.g5535a92b.el8.x8 9/25 2023-09-06T18:34:33.069 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:18.0.0-5801.g5535a92b.el8.x86_64 10/25 2023-09-06T18:34:33.070 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:18.0.0-5801.g5535a92b.el8.x86_64 11/25 2023-09-06T18:34:33.070 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:18.0.0-5801.g5535a92b.el8.x86_64 12/25 2023-09-06T18:34:33.070 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:18.0.0-5801.g5535a92b.el8.x86_64 13/25 2023-09-06T18:34:33.070 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-09-06T18:34:33.071 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-09-06T18:34:33.071 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-09-06T18:34:33.071 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 17/25 2023-09-06T18:34:33.071 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-09-06T18:34:33.072 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-09-06T18:34:33.072 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-09-06T18:34:33.072 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-09-06T18:34:33.073 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:18.0.0-5801.g5535a92b.el8.x86_64 22/25 2023-09-06T18:34:33.073 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-09-06T18:34:33.073 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:18.0.0-5801.g5535a92b.el8.x86_64 24/25 2023-09-06T18:34:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : jq-1.6-7.el8.x86_64 4/6 2023-09-06T18:34:33.389 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:18.0.0-5801.g5535a92b.el8.x86_64 5/6 2023-09-06T18:34:34.257 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-09-06T18:34:34.258 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:34:34.258 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2023-09-06T18:34:34.258 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.258 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.259 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:34:34.259 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.259 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.259 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.260 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.260 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-06T18:34:34.260 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.260 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-06T18:34:34.260 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-09-06T18:34:34.260 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.261 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-06T18:34:34.261 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.261 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-06T18:34:34.261 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-06T18:34:34.261 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.262 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.262 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.262 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-06T18:34:34.262 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.262 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.263 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:34.263 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-06T18:34:34.263 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:34:34.263 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:34:34.546 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-test 2023-09-06T18:34:35.393 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:45 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:34:36.940 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:34:36.940 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:34:36.941 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-09-06T18:34:36.941 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:34:36.941 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:34:36.941 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 48 M 2023-09-06T18:34:36.941 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-09-06T18:34:36.942 INFO:teuthology.orchestra.run.smithi181.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-09-06T18:34:36.942 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 139 k 2023-09-06T18:34:36.942 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-09-06T18:34:36.942 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-09-06T18:34:36.942 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-09-06T18:34:36.943 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:34:36.943 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:34:36.943 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:34:36.943 INFO:teuthology.orchestra.run.smithi181.stdout:Install 6 Packages 2023-09-06T18:34:36.943 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:34:36.944 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 49 M 2023-09-06T18:34:36.944 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 218 M 2023-09-06T18:34:36.945 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:34:37.773 INFO:teuthology.orchestra.run.smithi181.stdout:(1/6): libcephsqlite-18.0.0-5801.g5535a92b.el8. 186 kB/s | 139 kB 00:00 2023-09-06T18:34:38.049 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5801.g5535a92b.el8.x86_64 5/6 2023-09-06T18:34:38.265 INFO:teuthology.orchestra.run.smithi181.stdout:(2/6): jq-1.6-7.el8.x86_64.rpm 163 kB/s | 202 kB 00:01 2023-09-06T18:34:38.270 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:18.0.0-5801.g5535a92b.el8.x86_64 6/6 2023-09-06T18:34:38.582 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-test-2:18.0.0-5801.g5535a92b.el8.x86_64 6/6 2023-09-06T18:34:38.582 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:18.0.0-5801.g5535a92b.el8.x86_64 1/6 2023-09-06T18:34:38.582 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:18.0.0-5801.g5535a92b.el8.x86_64 2/6 2023-09-06T18:34:38.583 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : jq-1.6-7.el8.x86_64 3/6 2023-09-06T18:34:38.583 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-09-06T18:34:38.583 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-09-06T18:34:38.707 INFO:teuthology.orchestra.run.smithi181.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 201 kB/s | 187 kB 00:00 2023-09-06T18:34:39.268 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-09-06T18:34:39.269 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:34:39.269 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:34:39.269 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:39.269 INFO:teuthology.orchestra.run.smithi099.stdout: jq-1.6-7.el8.x86_64 2023-09-06T18:34:39.270 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:39.270 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-06T18:34:39.270 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-06T18:34:39.270 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-06T18:34:39.271 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:34:39.271 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:34:39.299 INFO:teuthology.orchestra.run.smithi181.stdout:(4/6): ceph-test-18.0.0-5801.g5535a92b.el8.x86_ 21 MB/s | 48 MB 00:02 2023-09-06T18:34:39.324 INFO:teuthology.orchestra.run.smithi181.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 114 kB/s | 71 kB 00:00 2023-09-06T18:34:39.547 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph 2023-09-06T18:34:39.783 INFO:teuthology.orchestra.run.smithi181.stdout:(6/6): socat-1.7.4.1-1.el8.x86_64.rpm 213 kB/s | 323 kB 00:01 2023-09-06T18:34:39.785 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:34:39.785 INFO:teuthology.orchestra.run.smithi181.stdout:Total 17 MB/s | 49 MB 00:02 2023-09-06T18:34:39.829 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:34:39.850 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:34:39.850 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:34:40.171 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:34:40.174 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:34:40.400 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:59 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:34:40.674 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:34:40.898 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-09-06T18:34:41.133 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-09-06T18:34:41.264 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-09-06T18:34:41.346 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-09-06T18:34:41.543 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : jq-1.6-7.el8.x86_64 4/6 2023-09-06T18:34:41.672 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:18.0.0-5801.g5535a92b.el8.x86_64 5/6 2023-09-06T18:34:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:34:41.969 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:34:41.970 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-06T18:34:41.970 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:34:41.970 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:34:41.970 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 6.4 k 2023-09-06T18:34:41.970 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-09-06T18:34:41.971 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 2.3 M 2023-09-06T18:34:41.971 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 1.6 M 2023-09-06T18:34:41.971 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 242 k 2023-09-06T18:34:41.971 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 4.6 M 2023-09-06T18:34:41.971 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 18 M 2023-09-06T18:34:41.972 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-06T18:34:41.972 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-06T18:34:41.972 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-09-06T18:34:41.972 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-06T18:34:41.972 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-06T18:34:41.973 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-09-06T18:34:41.973 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-06T18:34:41.973 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-06T18:34:41.973 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-06T18:34:41.973 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-09-06T18:34:41.973 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-09-06T18:34:41.974 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-06T18:34:41.974 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-06T18:34:41.974 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-06T18:34:41.975 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-09-06T18:34:41.975 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-09-06T18:34:41.976 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-09-06T18:34:41.976 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-06T18:34:41.976 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-06T18:34:41.976 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-06T18:34:41.976 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-06T18:34:41.977 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-06T18:34:41.977 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-06T18:34:41.977 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-06T18:34:41.977 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-09-06T18:34:41.977 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-06T18:34:41.978 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:34:41.978 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:34:41.978 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:34:41.978 INFO:teuthology.orchestra.run.smithi099.stdout:Install 32 Packages 2023-09-06T18:34:41.978 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:34:41.979 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 30 M 2023-09-06T18:34:41.979 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 120 M 2023-09-06T18:34:41.979 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:34:42.785 INFO:teuthology.orchestra.run.smithi099.stdout:(1/32): ceph-18.0.0-5801.g5535a92b.el8.x86_64.r 23 kB/s | 6.4 kB 00:00 2023-09-06T18:34:43.902 INFO:teuthology.orchestra.run.smithi099.stdout:(2/32): ceph-mgr-18.0.0-5801.g5535a92b.el8.x86_ 1.1 MB/s | 1.6 MB 00:01 2023-09-06T18:34:44.707 INFO:teuthology.orchestra.run.smithi099.stdout:(3/32): ceph-mds-18.0.0-5801.g5535a92b.el8.x86_ 1.0 MB/s | 2.3 MB 00:02 2023-09-06T18:34:44.946 INFO:teuthology.orchestra.run.smithi099.stdout:(4/32): ceph-mgr-modules-core-18.0.0-5801.g5535 1.0 MB/s | 242 kB 00:00 2023-09-06T18:34:45.658 INFO:teuthology.orchestra.run.smithi099.stdout:(5/32): ceph-mon-18.0.0-5801.g5535a92b.el8.x86_ 1.6 MB/s | 4.6 MB 00:02 2023-09-06T18:34:46.141 INFO:teuthology.orchestra.run.smithi099.stdout:(6/32): python3-mako-1.0.6-14.el8.noarch.rpm 131 kB/s | 157 kB 00:01 2023-09-06T18:34:46.158 INFO:teuthology.orchestra.run.smithi099.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 77 kB/s | 39 kB 00:00 2023-09-06T18:34:46.307 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5801.g5535a92b.el8.x86_64 5/6 2023-09-06T18:34:46.562 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:18.0.0-5801.g5535a92b.el8.x86_64 6/6 2023-09-06T18:34:46.792 INFO:teuthology.orchestra.run.smithi099.stdout:(8/32): python3-pytz-2017.2-11.el8.noarch.rpm 90 kB/s | 57 kB 00:00 2023-09-06T18:34:46.861 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-test-2:18.0.0-5801.g5535a92b.el8.x86_64 6/6 2023-09-06T18:34:46.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:18.0.0-5801.g5535a92b.el8.x86_64 1/6 2023-09-06T18:34:46.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:18.0.0-5801.g5535a92b.el8.x86_64 2/6 2023-09-06T18:34:46.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : jq-1.6-7.el8.x86_64 3/6 2023-09-06T18:34:46.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-09-06T18:34:46.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-09-06T18:34:47.076 INFO:teuthology.orchestra.run.smithi099.stdout:(9/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 111 kB/s | 103 kB 00:00 2023-09-06T18:34:47.251 INFO:teuthology.orchestra.run.smithi099.stdout:(10/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.4 MB/s | 249 kB 00:00 2023-09-06T18:34:47.334 INFO:teuthology.orchestra.run.smithi099.stdout:(11/32): python3-cryptography-3.2.1-6.el8.x86_6 7.5 MB/s | 641 kB 00:00 2023-09-06T18:34:47.418 INFO:teuthology.orchestra.run.smithi099.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 1.3 MB/s | 109 kB 00:00 2023-09-06T18:34:47.545 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-09-06T18:34:47.545 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:34:47.546 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:34:47.546 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:47.546 INFO:teuthology.orchestra.run.smithi181.stdout: jq-1.6-7.el8.x86_64 2023-09-06T18:34:47.546 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:34:47.546 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-06T18:34:47.547 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-06T18:34:47.547 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-06T18:34:47.547 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:34:47.547 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:34:47.751 INFO:teuthology.orchestra.run.smithi099.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 131 kB/s | 44 kB 00:00 2023-09-06T18:34:47.819 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph 2023-09-06T18:34:48.056 INFO:teuthology.orchestra.run.smithi099.stdout:(14/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 610 kB/s | 185 kB 00:00 2023-09-06T18:34:48.177 INFO:teuthology.orchestra.run.smithi099.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.4 MB/s | 173 kB 00:00 2023-09-06T18:34:48.369 INFO:teuthology.orchestra.run.smithi099.stdout:(16/32): python3-cherrypy-18.4.0-1.el8.noarch.r 2.0 MB/s | 384 kB 00:00 2023-09-06T18:34:48.477 INFO:teuthology.orchestra.run.smithi099.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-09-06T18:34:48.586 INFO:teuthology.orchestra.run.smithi099.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 162 kB/s | 18 kB 00:00 2023-09-06T18:34:48.677 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:58 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:34:48.695 INFO:teuthology.orchestra.run.smithi099.stdout:(19/32): python3-logutils-0.3.5-11.el8.noarch.r 450 kB/s | 49 kB 00:00 2023-09-06T18:34:48.779 INFO:teuthology.orchestra.run.smithi099.stdout:(20/32): python3-werkzeug-0.12.2-4.el8.noarch.r 230 kB/s | 457 kB 00:01 2023-09-06T18:34:48.804 INFO:teuthology.orchestra.run.smithi099.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 539 kB/s | 59 kB 00:00 2023-09-06T18:34:48.913 INFO:teuthology.orchestra.run.smithi099.stdout:(22/32): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2023-09-06T18:34:49.013 INFO:teuthology.orchestra.run.smithi099.stdout:(23/32): python3-simplegeneric-0.8.1-17.el8.noa 187 kB/s | 19 kB 00:00 2023-09-06T18:34:49.121 INFO:teuthology.orchestra.run.smithi099.stdout:(24/32): python3-singledispatch-3.4.0.3-18.el8. 221 kB/s | 24 kB 00:00 2023-09-06T18:34:49.222 INFO:teuthology.orchestra.run.smithi099.stdout:(25/32): python3-tempora-1.14.1-5.el8.noarch.rp 293 kB/s | 29 kB 00:00 2023-09-06T18:34:49.255 INFO:teuthology.orchestra.run.smithi099.stdout:(26/32): python3-pecan-1.3.2-9.el8.noarch.rpm 593 kB/s | 283 kB 00:00 2023-09-06T18:34:49.322 INFO:teuthology.orchestra.run.smithi099.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 275 kB/s | 27 kB 00:00 2023-09-06T18:34:49.389 INFO:teuthology.orchestra.run.smithi099.stdout:(28/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.9 MB/s | 255 kB 00:00 2023-09-06T18:34:49.464 INFO:teuthology.orchestra.run.smithi099.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.7 MB/s | 251 kB 00:00 2023-09-06T18:34:49.498 INFO:teuthology.orchestra.run.smithi099.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 788 kB/s | 85 kB 00:00 2023-09-06T18:34:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:(31/32): ceph-osd-18.0.0-5801.g5535a92b.el8.x86 3.0 MB/s | 18 MB 00:05 2023-09-06T18:34:49.848 INFO:teuthology.orchestra.run.smithi099.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 60 kB/s | 23 kB 00:00 2023-09-06T18:34:49.851 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:34:49.851 INFO:teuthology.orchestra.run.smithi099.stdout:Total 3.8 MB/s | 30 MB 00:07 2023-09-06T18:34:49.937 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:34:49.975 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:34:49.975 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:34:50.243 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:34:50.246 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:34:50.246 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-09-06T18:34:50.246 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:34:50.247 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:34:50.247 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 6.4 k 2023-09-06T18:34:50.247 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-09-06T18:34:50.247 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 2.3 M 2023-09-06T18:34:50.247 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 1.6 M 2023-09-06T18:34:50.247 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 242 k 2023-09-06T18:34:50.248 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 4.6 M 2023-09-06T18:34:50.248 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 18 M 2023-09-06T18:34:50.248 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-06T18:34:50.248 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-06T18:34:50.248 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-09-06T18:34:50.249 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-06T18:34:50.249 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-06T18:34:50.249 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-09-06T18:34:50.249 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-06T18:34:50.249 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-06T18:34:50.250 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-06T18:34:50.250 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-09-06T18:34:50.250 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-09-06T18:34:50.250 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-06T18:34:50.250 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-06T18:34:50.251 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-06T18:34:50.251 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-09-06T18:34:50.251 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-09-06T18:34:50.251 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-09-06T18:34:50.251 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-06T18:34:50.252 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-06T18:34:50.252 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-06T18:34:50.252 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-06T18:34:50.252 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-06T18:34:50.252 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-06T18:34:50.253 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-06T18:34:50.253 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-09-06T18:34:50.253 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-06T18:34:50.253 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:34:50.253 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:34:50.254 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:34:50.254 INFO:teuthology.orchestra.run.smithi181.stdout:Install 32 Packages 2023-09-06T18:34:50.254 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:34:50.255 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 30 M 2023-09-06T18:34:50.255 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 120 M 2023-09-06T18:34:50.255 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:34:50.292 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:34:50.295 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:34:51.084 INFO:teuthology.orchestra.run.smithi181.stdout:(1/32): ceph-18.0.0-5801.g5535a92b.el8.x86_64.r 30 kB/s | 6.4 kB 00:00 2023-09-06T18:34:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:34:51.699 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-09-06T18:34:51.967 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-09-06T18:34:52.001 INFO:teuthology.orchestra.run.smithi181.stdout:(2/32): ceph-mgr-18.0.0-5801.g5535a92b.el8.x86_ 1.4 MB/s | 1.6 MB 00:01 2023-09-06T18:34:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-09-06T18:34:52.434 INFO:teuthology.orchestra.run.smithi181.stdout:(3/32): ceph-mds-18.0.0-5801.g5535a92b.el8.x86_ 1.5 MB/s | 2.3 MB 00:01 2023-09-06T18:34:52.449 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2023-09-06T18:34:52.502 INFO:teuthology.orchestra.run.smithi181.stdout:(4/32): ceph-mon-18.0.0-5801.g5535a92b.el8.x86_ 3.3 MB/s | 4.6 MB 00:01 2023-09-06T18:34:52.568 INFO:teuthology.orchestra.run.smithi181.stdout:(5/32): ceph-mgr-modules-core-18.0.0-5801.g5535 1.8 MB/s | 242 kB 00:00 2023-09-06T18:34:52.690 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-09-06T18:34:52.893 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-09-06T18:34:53.104 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-09-06T18:34:53.152 INFO:teuthology.orchestra.run.smithi181.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 66 kB/s | 39 kB 00:00 2023-09-06T18:34:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-09-06T18:34:53.444 INFO:teuthology.orchestra.run.smithi181.stdout:(7/32): python3-mako-1.0.6-14.el8.noarch.rpm 167 kB/s | 157 kB 00:00 2023-09-06T18:34:53.558 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-09-06T18:34:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-09-06T18:34:54.059 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-09-06T18:34:54.169 INFO:teuthology.orchestra.run.smithi181.stdout:(8/32): ceph-osd-18.0.0-5801.g5535a92b.el8.x86_ 8.2 MB/s | 18 MB 00:02 2023-09-06T18:34:54.194 INFO:teuthology.orchestra.run.smithi181.stdout:(9/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 99 kB/s | 103 kB 00:01 2023-09-06T18:34:54.219 INFO:teuthology.orchestra.run.smithi181.stdout:(10/32): python3-pytz-2017.2-11.el8.noarch.rpm 73 kB/s | 57 kB 00:00 2023-09-06T18:34:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-09-06T18:34:54.726 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2023-09-06T18:34:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 14/32 2023-09-06T18:34:55.208 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-09-06T18:34:55.442 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-09-06T18:34:55.651 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-09-06T18:34:56.010 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-09-06T18:34:56.103 INFO:teuthology.orchestra.run.smithi181.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 237 kB/s | 457 kB 00:01 2023-09-06T18:34:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-09-06T18:34:56.345 INFO:teuthology.orchestra.run.smithi181.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 451 kB/s | 109 kB 00:00 2023-09-06T18:34:56.427 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2023-09-06T18:34:56.570 INFO:teuthology.orchestra.run.smithi181.stdout:(13/32): python3-cryptography-3.2.1-6.el8.x86_6 273 kB/s | 641 kB 00:02 2023-09-06T18:34:56.695 INFO:teuthology.orchestra.run.smithi181.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 124 kB/s | 44 kB 00:00 2023-09-06T18:34:57.012 INFO:teuthology.orchestra.run.smithi181.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 545 kB/s | 173 kB 00:00 2023-09-06T18:34:57.071 INFO:teuthology.orchestra.run.smithi181.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 370 kB/s | 185 kB 00:00 2023-09-06T18:34:57.188 INFO:teuthology.orchestra.run.smithi181.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-09-06T18:34:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-09-06T18:34:57.313 INFO:teuthology.orchestra.run.smithi181.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 141 kB/s | 18 kB 00:00 2023-09-06T18:34:57.438 INFO:teuthology.orchestra.run.smithi181.stdout:(19/32): python3-logutils-0.3.5-11.el8.noarch.r 390 kB/s | 49 kB 00:00 2023-09-06T18:34:57.522 INFO:teuthology.orchestra.run.smithi181.stdout:(20/32): python3-cherrypy-18.4.0-1.el8.noarch.r 755 kB/s | 384 kB 00:00 2023-09-06T18:34:57.564 INFO:teuthology.orchestra.run.smithi181.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 469 kB/s | 59 kB 00:00 2023-09-06T18:34:57.681 INFO:teuthology.orchestra.run.smithi181.stdout:(22/32): python3-portend-2.6-1.el8.noarch.rpm 133 kB/s | 16 kB 00:00 2023-09-06T18:34:57.764 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-09-06T18:34:57.806 INFO:teuthology.orchestra.run.smithi181.stdout:(23/32): python3-simplegeneric-0.8.1-17.el8.noa 150 kB/s | 19 kB 00:00 2023-09-06T18:34:57.832 INFO:teuthology.orchestra.run.smithi181.stdout:(24/32): python3-pecan-1.3.2-9.el8.noarch.rpm 915 kB/s | 283 kB 00:00 2023-09-06T18:34:57.932 INFO:teuthology.orchestra.run.smithi181.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 191 kB/s | 24 kB 00:00 2023-09-06T18:34:57.957 INFO:teuthology.orchestra.run.smithi181.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 233 kB/s | 29 kB 00:00 2023-09-06T18:34:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-09-06T18:34:58.066 INFO:teuthology.orchestra.run.smithi181.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 206 kB/s | 27 kB 00:00 2023-09-06T18:34:58.239 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-09-06T18:34:58.291 INFO:teuthology.orchestra.run.smithi181.stdout:(28/32): python3-waitress-1.4.3-1.el8.noarch.rp 766 kB/s | 255 kB 00:00 2023-09-06T18:34:58.316 INFO:teuthology.orchestra.run.smithi181.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.0 MB/s | 251 kB 00:00 2023-09-06T18:34:58.425 INFO:teuthology.orchestra.run.smithi181.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 639 kB/s | 85 kB 00:00 2023-09-06T18:34:58.450 INFO:teuthology.orchestra.run.smithi181.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 172 kB/s | 23 kB 00:00 2023-09-06T18:34:58.578 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2023-09-06T18:34:58.931 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-09-06T18:34:59.302 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5801.g5535a92b.el8. 27/32 2023-09-06T18:34:59.400 INFO:teuthology.orchestra.run.smithi181.stdout:(32/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 48 kB/s | 249 kB 00:05 2023-09-06T18:34:59.402 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:34:59.403 INFO:teuthology.orchestra.run.smithi181.stdout:Total 3.3 MB/s | 30 MB 00:09 2023-09-06T18:34:59.490 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:34:59.527 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:34:59.527 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:34:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:18.0.0-5801.g5535a92b.el8.x86_64 28/32 2023-09-06T18:34:59.850 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:34:59.853 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:35:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:35:01.120 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5801.g5535a92b.el8.x86_64 28/32 2023-09-06T18:35:01.249 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-09-06T18:35:01.313 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:18.0.0-5801.g5535a92b.el8.x86_64 29/32 2023-09-06T18:35:01.485 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-09-06T18:35:01.703 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-09-06T18:35:01.771 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:18.0.0-5801.g5535a92b.el8.x86_64 29/32 2023-09-06T18:35:01.950 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2023-09-06T18:35:01.999 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:18.0.0-5801.g5535a92b.el8.x86_64 30/32 2023-09-06T18:35:02.209 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-09-06T18:35:02.216 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:18.0.0-5801.g5535a92b.el8.x86_64 30/32 2023-09-06T18:35:02.374 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:18.0.0-5801.g5535a92b.el8.x86_64 31/32 2023-09-06T18:35:02.392 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:18.0.0-5801.g5535a92b.el8.x86_64 31/32 2023-09-06T18:35:02.437 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-09-06T18:35:02.499 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:18.0.0-5801.g5535a92b.el8.x86_64 32/32 2023-09-06T18:35:02.655 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-09-06T18:35:02.883 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-09-06T18:35:02.935 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-2:18.0.0-5801.g5535a92b.el8.x86_64 32/32 2023-09-06T18:35:02.935 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:18.0.0-5801.g5535a92b.el8.x86_64 1/32 2023-09-06T18:35:02.935 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:18.0.0-5801.g5535a92b.el8.x86_64 2/32 2023-09-06T18:35:02.935 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:18.0.0-5801.g5535a92b.el8.x86_64 3/32 2023-09-06T18:35:02.936 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:18.0.0-5801.g5535a92b.el8.x86_64 4/32 2023-09-06T18:35:02.936 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:18.0.0-5801.g5535a92b.el8.x86_64 5/32 2023-09-06T18:35:02.936 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5801.g5535a92b.el8. 6/32 2023-09-06T18:35:02.936 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2023-09-06T18:35:02.936 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-09-06T18:35:02.937 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-09-06T18:35:02.937 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2023-09-06T18:35:02.937 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-09-06T18:35:02.937 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2023-09-06T18:35:02.937 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 13/32 2023-09-06T18:35:02.938 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-09-06T18:35:02.938 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-09-06T18:35:02.938 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-09-06T18:35:02.938 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-09-06T18:35:02.939 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-09-06T18:35:02.939 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-09-06T18:35:02.939 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-09-06T18:35:02.939 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-09-06T18:35:02.939 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-09-06T18:35:02.939 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-09-06T18:35:02.940 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-09-06T18:35:02.940 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-09-06T18:35:02.940 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-09-06T18:35:02.941 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-09-06T18:35:02.942 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-09-06T18:35:02.942 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2023-09-06T18:35:02.943 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-09-06T18:35:02.943 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-09-06T18:35:03.092 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-09-06T18:35:03.330 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-09-06T18:35:03.526 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-09-06T18:35:03.800 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-09-06T18:35:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-09-06T18:35:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:03.978 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:35:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:35:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:35:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:35:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:35:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:35:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:35:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-06T18:35:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-06T18:35:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-09-06T18:35:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-06T18:35:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-06T18:35:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-09-06T18:35:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-06T18:35:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-06T18:35:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-06T18:35:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.0.6-14.el8.noarch 2023-09-06T18:35:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-06T18:35:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-06T18:35:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-06T18:35:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-2.6-1.el8.noarch 2023-09-06T18:35:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-06T18:35:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-06T18:35:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2017.2-11.el8.noarch 2023-09-06T18:35:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-06T18:35:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-06T18:35:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-06T18:35:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-06T18:35:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-06T18:35:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-06T18:35:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-06T18:35:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-06T18:35:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-06T18:35:03.985 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:03.985 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:35:04.218 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2023-09-06T18:35:04.236 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-base 2023-09-06T18:35:04.513 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 14/32 2023-09-06T18:35:04.726 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-09-06T18:35:04.958 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-09-06T18:35:05.085 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:24 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:35:05.211 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-09-06T18:35:05.527 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-09-06T18:35:06.277 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-09-06T18:35:06.474 INFO:teuthology.orchestra.run.smithi099.stdout:Package ceph-base-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:35:06.497 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2023-09-06T18:35:06.650 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:35:06.651 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2023-09-06T18:35:06.651 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:35:06.699 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-09-06T18:35:06.753 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install cephadm 2023-09-06T18:35:06.964 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-09-06T18:35:07.202 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-09-06T18:35:07.456 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-09-06T18:35:07.596 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:26 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:35:07.762 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2023-09-06T18:35:08.058 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-09-06T18:35:08.434 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5801.g5535a92b.el8. 27/32 2023-09-06T18:35:08.644 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:18.0.0-5801.g5535a92b.el8.x86_64 28/32 2023-09-06T18:35:09.161 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:35:09.161 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:35:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-06T18:35:09.162 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:35:09.162 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:35:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 174 k 2023-09-06T18:35:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:09.163 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:35:09.163 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:35:09.163 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2023-09-06T18:35:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:09.164 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 174 k 2023-09-06T18:35:09.164 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 694 k 2023-09-06T18:35:09.164 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:35:09.644 INFO:teuthology.orchestra.run.smithi099.stdout:cephadm-18.0.0-5801.g5535a92b.el8.noarch.rpm 361 kB/s | 174 kB 00:00 2023-09-06T18:35:09.644 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:35:09.645 INFO:teuthology.orchestra.run.smithi099.stdout:Total 360 kB/s | 174 kB 00:00 2023-09-06T18:35:09.645 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:35:09.652 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:35:09.652 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:35:09.654 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:35:09.657 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:35:09.698 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:35:10.191 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 1/1 2023-09-06T18:35:10.262 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5801.g5535a92b.el8.x86_64 28/32 2023-09-06T18:35:10.365 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 1/1 2023-09-06T18:35:10.419 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:18.0.0-5801.g5535a92b.el8.x86_64 29/32 2023-09-06T18:35:10.588 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 1/1 2023-09-06T18:35:10.880 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:18.0.0-5801.g5535a92b.el8.x86_64 29/32 2023-09-06T18:35:11.093 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:18.0.0-5801.g5535a92b.el8.x86_64 30/32 2023-09-06T18:35:11.179 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 1/1 2023-09-06T18:35:11.179 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:11.179 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:35:11.179 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:35:11.180 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:11.180 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:35:11.309 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:18.0.0-5801.g5535a92b.el8.x86_64 30/32 2023-09-06T18:35:11.400 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-immutable-object-cache 2023-09-06T18:35:11.484 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:18.0.0-5801.g5535a92b.el8.x86_64 31/32 2023-09-06T18:35:11.503 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:18.0.0-5801.g5535a92b.el8.x86_64 31/32 2023-09-06T18:35:11.626 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:18.0.0-5801.g5535a92b.el8.x86_64 32/32 2023-09-06T18:35:12.059 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-2:18.0.0-5801.g5535a92b.el8.x86_64 32/32 2023-09-06T18:35:12.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:18.0.0-5801.g5535a92b.el8.x86_64 1/32 2023-09-06T18:35:12.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:18.0.0-5801.g5535a92b.el8.x86_64 2/32 2023-09-06T18:35:12.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:18.0.0-5801.g5535a92b.el8.x86_64 3/32 2023-09-06T18:35:12.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:18.0.0-5801.g5535a92b.el8.x86_64 4/32 2023-09-06T18:35:12.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:18.0.0-5801.g5535a92b.el8.x86_64 5/32 2023-09-06T18:35:12.061 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5801.g5535a92b.el8. 6/32 2023-09-06T18:35:12.061 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2023-09-06T18:35:12.061 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-09-06T18:35:12.061 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-09-06T18:35:12.062 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2023-09-06T18:35:12.062 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-09-06T18:35:12.062 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2023-09-06T18:35:12.062 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 13/32 2023-09-06T18:35:12.062 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-09-06T18:35:12.063 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-09-06T18:35:12.063 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-09-06T18:35:12.063 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-09-06T18:35:12.063 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-09-06T18:35:12.063 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-09-06T18:35:12.064 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-09-06T18:35:12.064 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-09-06T18:35:12.064 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-09-06T18:35:12.064 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-09-06T18:35:12.064 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-09-06T18:35:12.065 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-09-06T18:35:12.065 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-09-06T18:35:12.065 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-09-06T18:35:12.065 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-09-06T18:35:12.066 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2023-09-06T18:35:12.066 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-09-06T18:35:12.066 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-09-06T18:35:12.243 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:31 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:35:13.203 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-09-06T18:35:13.203 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:13.204 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:35:13.204 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:35:13.204 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:35:13.204 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:35:13.204 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:35:13.205 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:35:13.205 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:35:13.205 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-06T18:35:13.205 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-06T18:35:13.205 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-09-06T18:35:13.206 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-06T18:35:13.206 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-06T18:35:13.206 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-09-06T18:35:13.206 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-06T18:35:13.206 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-06T18:35:13.207 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-06T18:35:13.207 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.0.6-14.el8.noarch 2023-09-06T18:35:13.207 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-06T18:35:13.207 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-06T18:35:13.207 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-06T18:35:13.208 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-2.6-1.el8.noarch 2023-09-06T18:35:13.208 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-06T18:35:13.208 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-06T18:35:13.208 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2017.2-11.el8.noarch 2023-09-06T18:35:13.208 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-06T18:35:13.208 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-06T18:35:13.209 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-06T18:35:13.209 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-06T18:35:13.209 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-06T18:35:13.209 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-06T18:35:13.209 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-06T18:35:13.210 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-06T18:35:13.210 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-06T18:35:13.210 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:13.210 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:35:13.441 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-base 2023-09-06T18:35:13.793 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:35:13.794 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:35:13.794 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repo Size 2023-09-06T18:35:13.794 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:35:13.795 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:35:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 167 k 2023-09-06T18:35:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:13.795 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:35:13.795 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:35:13.796 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2023-09-06T18:35:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:13.796 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 167 k 2023-09-06T18:35:13.796 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 512 k 2023-09-06T18:35:13.796 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:35:14.224 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-immutable-object-cache-18.0.0-5801.g5535a9 391 kB/s | 167 kB 00:00 2023-09-06T18:35:14.224 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:35:14.225 INFO:teuthology.orchestra.run.smithi099.stdout:Total 390 kB/s | 167 kB 00:00 2023-09-06T18:35:14.225 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:35:14.237 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:35:14.237 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:35:14.282 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:35:14.285 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:35:14.294 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:24 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:35:14.402 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:35:14.597 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5801.g5535a92b. 1/1 2023-09-06T18:35:15.028 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5801.g5535a92b. 1/1 2023-09-06T18:35:15.654 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5801.g5535a92b. 1/1 2023-09-06T18:35:15.654 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:15.655 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:35:15.655 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:35:15.655 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:15.655 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:35:15.662 INFO:teuthology.orchestra.run.smithi181.stdout:Package ceph-base-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:35:15.837 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:35:15.838 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2023-09-06T18:35:15.838 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:35:15.844 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-mgr 2023-09-06T18:35:15.953 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install cephadm 2023-09-06T18:35:16.698 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:35 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:35:16.804 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:26 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:35:18.066 INFO:teuthology.orchestra.run.smithi099.stdout:Package ceph-mgr-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:35:18.240 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:35:18.241 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2023-09-06T18:35:18.242 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:35:18.353 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-mgr-dashboard 2023-09-06T18:35:18.367 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:35:18.367 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:35:18.368 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-09-06T18:35:18.368 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:35:18.368 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:35:18.368 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 174 k 2023-09-06T18:35:18.369 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:18.369 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:35:18.369 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:35:18.369 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2023-09-06T18:35:18.369 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:18.370 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 174 k 2023-09-06T18:35:18.370 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 694 k 2023-09-06T18:35:18.370 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:35:18.924 INFO:teuthology.orchestra.run.smithi181.stdout:cephadm-18.0.0-5801.g5535a92b.el8.noarch.rpm 313 kB/s | 174 kB 00:00 2023-09-06T18:35:18.924 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:35:18.925 INFO:teuthology.orchestra.run.smithi181.stdout:Total 313 kB/s | 174 kB 00:00 2023-09-06T18:35:18.925 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:35:18.932 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:35:18.932 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:35:18.934 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:35:18.937 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:35:18.985 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:35:19.193 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:38 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:35:19.535 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 1/1 2023-09-06T18:35:19.680 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 1/1 2023-09-06T18:35:19.909 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 1/1 2023-09-06T18:35:20.525 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 1/1 2023-09-06T18:35:20.525 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:20.525 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:35:20.526 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:35:20.526 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:20.526 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:35:20.743 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:35:20.745 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:35:20.745 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-06T18:35:20.745 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:35:20.745 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:35:20.745 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 1.7 M 2023-09-06T18:35:20.746 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-09-06T18:35:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 23 k 2023-09-06T18:35:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 15 k 2023-09-06T18:35:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-09-06T18:35:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-06T18:35:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-06T18:35:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:20.747 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:35:20.748 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:35:20.748 INFO:teuthology.orchestra.run.smithi099.stdout:Install 6 Packages 2023-09-06T18:35:20.748 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:20.749 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 2.1 M 2023-09-06T18:35:20.749 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 8.5 M 2023-09-06T18:35:20.749 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:35:20.777 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-immutable-object-cache 2023-09-06T18:35:21.009 INFO:teuthology.orchestra.run.smithi099.stdout:(1/6): ceph-grafana-dashboards-18.0.0-5801.g553 156 kB/s | 23 kB 00:00 2023-09-06T18:35:21.026 INFO:teuthology.orchestra.run.smithi099.stdout:(2/6): ceph-prometheus-alerts-18.0.0-5801.g5535 88 kB/s | 15 kB 00:00 2023-09-06T18:35:21.144 INFO:teuthology.orchestra.run.smithi099.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 318 kB/s | 43 kB 00:00 2023-09-06T18:35:21.185 INFO:teuthology.orchestra.run.smithi099.stdout:(4/6): ceph-mgr-dashboard-18.0.0-5801.g5535a92b 5.4 MB/s | 1.7 MB 00:00 2023-09-06T18:35:21.302 INFO:teuthology.orchestra.run.smithi099.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 122 kB/s | 34 kB 00:00 2023-09-06T18:35:21.494 INFO:teuthology.orchestra.run.smithi099.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 562 kB/s | 196 kB 00:00 2023-09-06T18:35:21.497 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:35:21.497 INFO:teuthology.orchestra.run.smithi099.stdout:Total 2.7 MB/s | 2.1 MB 00:00 2023-09-06T18:35:21.507 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:35:21.516 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:35:21.516 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:35:21.589 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:35:21.592 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:35:21.622 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:31 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:35:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:35:22.058 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-09-06T18:35:22.278 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-09-06T18:35:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-09-06T18:35:22.600 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5801.g5535a92b.el8.n 4/6 2023-09-06T18:35:22.928 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5801.g5535a92b.el8. 5/6 2023-09-06T18:35:23.179 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:35:23.179 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:35:23.180 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repo Size 2023-09-06T18:35:23.180 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:35:23.180 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:35:23.180 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 167 k 2023-09-06T18:35:23.180 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:23.181 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:35:23.181 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:35:23.181 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2023-09-06T18:35:23.181 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:23.181 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 167 k 2023-09-06T18:35:23.182 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 512 k 2023-09-06T18:35:23.182 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:35:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5801.g5535a92b.el8.noarc 6/6 2023-09-06T18:35:23.361 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-immutable-object-cache-18.0.0-5801.g5535a9 926 kB/s | 167 kB 00:00 2023-09-06T18:35:23.362 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:35:23.362 INFO:teuthology.orchestra.run.smithi181.stdout:Total 920 kB/s | 167 kB 00:00 2023-09-06T18:35:23.363 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:35:23.373 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:35:23.373 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:35:23.419 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:35:23.422 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:35:23.477 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5801.g5535a92b.el8.noarc 6/6 2023-09-06T18:35:23.477 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5801.g5535a92b.el8. 1/6 2023-09-06T18:35:23.478 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5801.g5535a92b.el8.noarc 2/6 2023-09-06T18:35:23.478 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5801.g5535a92b.el8.n 3/6 2023-09-06T18:35:23.478 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-09-06T18:35:23.478 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-09-06T18:35:23.574 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:35:23.741 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5801.g5535a92b. 1/1 2023-09-06T18:35:24.109 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-09-06T18:35:24.109 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:24.110 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:35:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:35:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:35:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:35:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-06T18:35:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-06T18:35:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-06T18:35:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:24.112 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:35:24.196 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5801.g5535a92b. 1/1 2023-09-06T18:35:24.271 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-mgr-diskprediction-local 2023-09-06T18:35:24.784 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5801.g5535a92b. 1/1 2023-09-06T18:35:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:24.785 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:35:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:35:24.786 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:24.786 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:35:25.037 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr 2023-09-06T18:35:25.120 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:44 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:35:25.894 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:35 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:35:26.653 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:35:26.654 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================================ 2023-09-06T18:35:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-06T18:35:26.657 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================================ 2023-09-06T18:35:26.657 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:35:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 7.4 M 2023-09-06T18:35:26.658 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-09-06T18:35:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-09-06T18:35:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-09-06T18:35:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-09-06T18:35:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-09-06T18:35:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-09-06T18:35:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-09-06T18:35:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-09-06T18:35:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:26.660 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:35:26.660 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================================ 2023-09-06T18:35:26.660 INFO:teuthology.orchestra.run.smithi099.stdout:Install 8 Packages 2023-09-06T18:35:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:26.661 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 35 M 2023-09-06T18:35:26.661 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 210 M 2023-09-06T18:35:26.661 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:35:27.265 INFO:teuthology.orchestra.run.smithi181.stdout:Package ceph-mgr-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:35:27.443 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:35:27.444 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2023-09-06T18:35:27.444 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:35:27.554 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-dashboard 2023-09-06T18:35:28.191 INFO:teuthology.orchestra.run.smithi099.stdout:(1/8): ceph-mgr-diskprediction-local-18.0.0-580 5.7 MB/s | 7.4 MB 00:01 2023-09-06T18:35:28.410 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:38 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:35:29.972 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:35:29.978 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:35:29.978 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-09-06T18:35:29.979 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:35:29.979 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:35:29.979 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 1.7 M 2023-09-06T18:35:29.979 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-09-06T18:35:29.980 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 23 k 2023-09-06T18:35:29.980 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 15 k 2023-09-06T18:35:29.980 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-09-06T18:35:29.980 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-06T18:35:29.981 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-06T18:35:29.981 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:29.981 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:35:29.981 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:35:29.981 INFO:teuthology.orchestra.run.smithi181.stdout:Install 6 Packages 2023-09-06T18:35:29.981 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:29.982 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 2.1 M 2023-09-06T18:35:29.983 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 8.5 M 2023-09-06T18:35:29.983 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:35:30.373 INFO:teuthology.orchestra.run.smithi181.stdout:(1/6): ceph-prometheus-alerts-18.0.0-5801.g5535 97 kB/s | 15 kB 00:00 2023-09-06T18:35:30.506 INFO:teuthology.orchestra.run.smithi181.stdout:(2/6): python3-jwt-1.6.1-2.el8.noarch.rpm 320 kB/s | 43 kB 00:00 2023-09-06T18:35:30.532 INFO:teuthology.orchestra.run.smithi181.stdout:(3/6): ceph-grafana-dashboards-18.0.0-5801.g553 74 kB/s | 23 kB 00:00 2023-09-06T18:35:30.573 INFO:teuthology.orchestra.run.smithi181.stdout:(4/6): ceph-mgr-dashboard-18.0.0-5801.g5535a92b 5.0 MB/s | 1.7 MB 00:00 2023-09-06T18:35:30.932 INFO:teuthology.orchestra.run.smithi181.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 79 kB/s | 34 kB 00:00 2023-09-06T18:35:31.107 INFO:teuthology.orchestra.run.smithi181.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 341 kB/s | 196 kB 00:00 2023-09-06T18:35:31.110 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:35:31.111 INFO:teuthology.orchestra.run.smithi181.stdout:Total 1.8 MB/s | 2.1 MB 00:01 2023-09-06T18:35:31.123 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:35:31.133 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:35:31.133 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:35:31.211 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:35:31.215 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:35:31.473 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:35:31.653 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-09-06T18:35:31.915 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-09-06T18:35:32.092 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-09-06T18:35:32.246 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5801.g5535a92b.el8.n 4/6 2023-09-06T18:35:32.467 INFO:teuthology.orchestra.run.smithi099.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 891 kB/s | 4.9 MB 00:05 2023-09-06T18:35:32.555 INFO:teuthology.orchestra.run.smithi099.stdout:(3/8): openblas-0.3.15-4.el8.x86_64.rpm 854 kB/s | 4.7 MB 00:05 2023-09-06T18:35:32.576 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5801.g5535a92b.el8. 5/6 2023-09-06T18:35:32.773 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5801.g5535a92b.el8.noarc 6/6 2023-09-06T18:35:33.055 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5801.g5535a92b.el8.noarc 6/6 2023-09-06T18:35:33.055 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5801.g5535a92b.el8. 1/6 2023-09-06T18:35:33.055 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5801.g5535a92b.el8.noarc 2/6 2023-09-06T18:35:33.055 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5801.g5535a92b.el8.n 3/6 2023-09-06T18:35:33.056 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-09-06T18:35:33.056 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-09-06T18:35:33.674 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-09-06T18:35:33.674 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:33.675 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:35:33.675 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:35:33.676 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:35:33.676 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:35:33.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-06T18:35:33.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-06T18:35:33.677 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-06T18:35:33.677 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:33.677 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:35:33.710 INFO:teuthology.orchestra.run.smithi099.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 693 kB/s | 3.7 MB 00:05 2023-09-06T18:35:33.808 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-diskprediction-local 2023-09-06T18:35:33.868 INFO:teuthology.orchestra.run.smithi099.stdout:(5/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 161 kB/s | 225 kB 00:01 2023-09-06T18:35:33.939 INFO:teuthology.orchestra.run.smithi099.stdout:(6/8): libgfortran-8.5.0-20.el8.x86_64.rpm 2.8 MB/s | 645 kB 00:00 2023-09-06T18:35:34.019 INFO:teuthology.orchestra.run.smithi099.stdout:(7/8): libquadmath-8.5.0-20.el8.x86_64.rpm 1.1 MB/s | 172 kB 00:00 2023-09-06T18:35:34.656 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:44 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:35:36.213 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:35:36.214 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================================ 2023-09-06T18:35:36.215 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-09-06T18:35:36.215 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================================ 2023-09-06T18:35:36.215 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:35:36.215 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 7.4 M 2023-09-06T18:35:36.216 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-09-06T18:35:36.216 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-09-06T18:35:36.216 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-09-06T18:35:36.216 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-09-06T18:35:36.216 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-09-06T18:35:36.217 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-09-06T18:35:36.217 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-09-06T18:35:36.217 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-09-06T18:35:36.217 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:36.217 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:35:36.218 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================================ 2023-09-06T18:35:36.218 INFO:teuthology.orchestra.run.smithi181.stdout:Install 8 Packages 2023-09-06T18:35:36.218 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:36.219 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 35 M 2023-09-06T18:35:36.219 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 210 M 2023-09-06T18:35:36.219 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:35:37.722 INFO:teuthology.orchestra.run.smithi181.stdout:(1/8): ceph-mgr-diskprediction-local-18.0.0-580 6.6 MB/s | 7.4 MB 00:01 2023-09-06T18:35:38.310 INFO:teuthology.orchestra.run.smithi099.stdout:(8/8): python3-scipy-1.0.0-21.module_el8.5.0+77 2.4 MB/s | 14 MB 00:05 2023-09-06T18:35:38.311 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:35:38.312 INFO:teuthology.orchestra.run.smithi099.stdout:Total 3.0 MB/s | 35 MB 00:11 2023-09-06T18:35:38.547 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:35:38.565 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:35:38.565 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:35:38.986 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:35:38.989 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:35:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:35:39.724 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 1/8 2023-09-06T18:35:39.842 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 1/8 2023-09-06T18:35:39.998 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 2/8 2023-09-06T18:35:40.585 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 2/8 2023-09-06T18:35:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-09-06T18:35:41.209 INFO:teuthology.orchestra.run.smithi181.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 1.1 MB/s | 3.7 MB 00:03 2023-09-06T18:35:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-09-06T18:35:41.515 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-09-06T18:35:42.141 INFO:teuthology.orchestra.run.smithi181.stdout:(3/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 897 kB/s | 4.9 MB 00:05 2023-09-06T18:35:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-09-06T18:35:42.308 INFO:teuthology.orchestra.run.smithi181.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 205 kB/s | 225 kB 00:01 2023-09-06T18:35:42.524 INFO:teuthology.orchestra.run.smithi181.stdout:(5/8): libgfortran-8.5.0-20.el8.x86_64.rpm 2.9 MB/s | 645 kB 00:00 2023-09-06T18:35:42.592 INFO:teuthology.orchestra.run.smithi181.stdout:(6/8): libquadmath-8.5.0-20.el8.x86_64.rpm 2.5 MB/s | 172 kB 00:00 2023-09-06T18:35:42.602 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-09-06T18:35:42.857 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-09-06T18:35:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-09-06T18:35:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-09-06T18:35:46.139 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5801.g5535a92 8/8 2023-09-06T18:35:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5801.g5535a92 8/8 2023-09-06T18:35:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5801.g5535a92 1/8 2023-09-06T18:35:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-09-06T18:35:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-09-06T18:35:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-09-06T18:35:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-09-06T18:35:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-09-06T18:35:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 7/8 2023-09-06T18:35:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 8/8 2023-09-06T18:35:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:47.114 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:35:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:35:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-09-06T18:35:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-09-06T18:35:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.15-4.el8.x86_64 2023-09-06T18:35:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-09-06T18:35:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-06T18:35:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-06T18:35:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-09-06T18:35:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:47.117 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:35:47.266 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-mgr-rook 2023-09-06T18:35:48.117 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:07 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:35:48.242 INFO:teuthology.orchestra.run.smithi181.stdout:(7/8): python3-scipy-1.0.0-21.module_el8.5.0+77 2.2 MB/s | 14 MB 00:06 2023-09-06T18:35:49.668 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:35:49.669 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:35:49.670 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-06T18:35:49.670 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:35:49.671 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:35:49.671 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 47 k 2023-09-06T18:35:49.671 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-09-06T18:35:49.671 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-06T18:35:49.671 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-06T18:35:49.672 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-06T18:35:49.672 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-09-06T18:35:49.672 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-09-06T18:35:49.672 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-06T18:35:49.672 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-09-06T18:35:49.673 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-09-06T18:35:49.673 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-09-06T18:35:49.673 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-09-06T18:35:49.673 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-06T18:35:49.673 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-06T18:35:49.674 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:49.674 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:35:49.674 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:35:49.674 INFO:teuthology.orchestra.run.smithi099.stdout:Install 13 Packages 2023-09-06T18:35:49.674 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:49.675 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 1.9 M 2023-09-06T18:35:49.675 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 25 M 2023-09-06T18:35:49.676 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:35:50.098 INFO:teuthology.orchestra.run.smithi099.stdout:(1/13): ceph-mgr-rook-18.0.0-5801.g5535a92b.el8 306 kB/s | 47 kB 00:00 2023-09-06T18:35:50.373 INFO:teuthology.orchestra.run.smithi099.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 62 kB/s | 27 kB 00:00 2023-09-06T18:35:50.500 INFO:teuthology.orchestra.run.smithi181.stdout:(8/8): openblas-0.3.15-4.el8.x86_64.rpm 348 kB/s | 4.7 MB 00:13 2023-09-06T18:35:50.501 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:35:50.501 INFO:teuthology.orchestra.run.smithi181.stdout:Total 2.5 MB/s | 35 MB 00:14 2023-09-06T18:35:50.557 INFO:teuthology.orchestra.run.smithi099.stdout:(3/13): python3-jsonpointer-1.10-11.el8.noarch. 32 kB/s | 20 kB 00:00 2023-09-06T18:35:50.738 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:35:50.756 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:35:50.757 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:35:50.899 INFO:teuthology.orchestra.run.smithi099.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 158 kB/s | 126 kB 00:00 2023-09-06T18:35:50.916 INFO:teuthology.orchestra.run.smithi099.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 432 kB/s | 155 kB 00:00 2023-09-06T18:35:51.150 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:35:51.153 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:35:51.249 INFO:teuthology.orchestra.run.smithi099.stdout:(6/13): python3-pyasn1-modules-0.3.7-6.el8.noar 126 kB/s | 110 kB 00:00 2023-09-06T18:35:51.341 INFO:teuthology.orchestra.run.smithi099.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 96 kB/s | 43 kB 00:00 2023-09-06T18:35:51.366 INFO:teuthology.orchestra.run.smithi099.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 73 kB/s | 33 kB 00:00 2023-09-06T18:35:51.684 INFO:teuthology.orchestra.run.smithi099.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 38 kB/s | 16 kB 00:00 2023-09-06T18:35:51.741 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:35:51.869 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 1/8 2023-09-06T18:35:51.900 INFO:teuthology.orchestra.run.smithi099.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 147 kB/s | 81 kB 00:00 2023-09-06T18:35:51.942 INFO:teuthology.orchestra.run.smithi099.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 248 kB/s | 64 kB 00:00 2023-09-06T18:35:52.003 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 1/8 2023-09-06T18:35:52.009 INFO:teuthology.orchestra.run.smithi099.stdout:(12/13): python3-kubernetes-11.0.0-6.el8.noarch 1.8 MB/s | 1.2 MB 00:00 2023-09-06T18:35:52.051 INFO:teuthology.orchestra.run.smithi099.stdout:(13/13): python3-websocket-client-0.56.0-5.el8. 408 kB/s | 61 kB 00:00 2023-09-06T18:35:52.054 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:35:52.054 INFO:teuthology.orchestra.run.smithi099.stdout:Total 830 kB/s | 1.9 MB 00:02 2023-09-06T18:35:52.114 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:35:52.130 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:35:52.131 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:35:52.177 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 2/8 2023-09-06T18:35:52.217 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:35:52.220 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:35:52.739 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:35:52.756 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 2/8 2023-09-06T18:35:52.944 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-09-06T18:35:53.005 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-09-06T18:35:53.221 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-09-06T18:35:53.481 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2023-09-06T18:35:53.543 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-09-06T18:35:53.719 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-09-06T18:35:53.736 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-09-06T18:35:53.943 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-09-06T18:35:54.225 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-09-06T18:35:54.404 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-09-06T18:35:54.566 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-09-06T18:35:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-09-06T18:35:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-09-06T18:35:55.069 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-09-06T18:35:55.896 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-09-06T18:35:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-09-06T18:35:56.558 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-09-06T18:35:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-09-06T18:35:56.828 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-09-06T18:35:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:18.0.0-5801.g5535a92b.el8.noarch 13/13 2023-09-06T18:35:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5801.g5535a92b.el8.noarch 13/13 2023-09-06T18:35:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5801.g5535a92b.el8.noarch 1/13 2023-09-06T18:35:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-09-06T18:35:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-09-06T18:35:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-09-06T18:35:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-09-06T18:35:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-09-06T18:35:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-09-06T18:35:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-09-06T18:35:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-09-06T18:35:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-09-06T18:35:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-09-06T18:35:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2023-09-06T18:35:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-09-06T18:35:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:57.988 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:35:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:35:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-06T18:35:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-06T18:35:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-06T18:35:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-06T18:35:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-06T18:35:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-06T18:35:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-06T18:35:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-06T18:35:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-06T18:35:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-06T18:35:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-06T18:35:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-06T18:35:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:35:57.991 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:35:58.095 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-09-06T18:35:58.107 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-mgr-cephadm 2023-09-06T18:35:58.335 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5801.g5535a92 8/8 2023-09-06T18:35:58.657 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5801.g5535a92 8/8 2023-09-06T18:35:58.657 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5801.g5535a92 1/8 2023-09-06T18:35:58.657 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-09-06T18:35:58.657 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-09-06T18:35:58.657 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-09-06T18:35:58.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-09-06T18:35:58.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-09-06T18:35:58.658 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 7/8 2023-09-06T18:35:58.947 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:17 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:35:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 8/8 2023-09-06T18:35:59.304 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:59.304 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:35:59.305 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:35:59.305 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-09-06T18:35:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-09-06T18:35:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.15-4.el8.x86_64 2023-09-06T18:35:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-09-06T18:35:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-06T18:35:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-06T18:35:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-09-06T18:35:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:35:59.307 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:35:59.437 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-rook 2023-09-06T18:36:00.287 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:10 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:36:00.501 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:00.502 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-06T18:36:00.503 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:00.503 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:36:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 132 k 2023-09-06T18:36:00.503 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-09-06T18:36:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-06T18:36:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-09-06T18:36:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-09-06T18:36:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-06T18:36:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:00.505 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:36:00.505 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:00.505 INFO:teuthology.orchestra.run.smithi099.stdout:Install 5 Packages 2023-09-06T18:36:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:00.506 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 6.0 M 2023-09-06T18:36:00.506 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 26 M 2023-09-06T18:36:00.506 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:36:01.174 INFO:teuthology.orchestra.run.smithi099.stdout:(1/5): ceph-mgr-cephadm-18.0.0-5801.g5535a92b.e 418 kB/s | 132 kB 00:00 2023-09-06T18:36:01.458 INFO:teuthology.orchestra.run.smithi099.stdout:(2/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.6 MB/s | 476 kB 00:00 2023-09-06T18:36:01.508 INFO:teuthology.orchestra.run.smithi099.stdout:(3/5): python3-natsort-7.1.1-2.el8.noarch.rpm 1.2 MB/s | 60 kB 00:00 2023-09-06T18:36:01.844 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:36:01.845 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:01.846 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-09-06T18:36:01.846 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:01.847 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:36:01.847 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 47 k 2023-09-06T18:36:01.847 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-09-06T18:36:01.848 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-06T18:36:01.848 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-06T18:36:01.848 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-06T18:36:01.848 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-09-06T18:36:01.849 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-09-06T18:36:01.849 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-06T18:36:01.849 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-09-06T18:36:01.849 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-09-06T18:36:01.850 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-09-06T18:36:01.850 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-09-06T18:36:01.851 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-06T18:36:01.851 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-06T18:36:01.851 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:01.852 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:36:01.852 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:01.852 INFO:teuthology.orchestra.run.smithi181.stdout:Install 13 Packages 2023-09-06T18:36:01.852 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:01.853 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 1.9 M 2023-09-06T18:36:01.854 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 25 M 2023-09-06T18:36:01.854 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:36:02.375 INFO:teuthology.orchestra.run.smithi099.stdout:(4/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 355 kB/s | 538 kB 00:01 2023-09-06T18:36:03.525 INFO:teuthology.orchestra.run.smithi181.stdout:(1/13): ceph-mgr-rook-18.0.0-5801.g5535a92b.el8 169 kB/s | 47 kB 00:00 2023-09-06T18:36:03.641 INFO:teuthology.orchestra.run.smithi181.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 50 kB/s | 20 kB 00:00 2023-09-06T18:36:03.658 INFO:teuthology.orchestra.run.smithi181.stdout:(3/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 64 kB/s | 27 kB 00:00 2023-09-06T18:36:03.809 INFO:teuthology.orchestra.run.smithi181.stdout:(4/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.0 MB/s | 155 kB 00:00 2023-09-06T18:36:04.067 INFO:teuthology.orchestra.run.smithi181.stdout:(5/13): python3-requests-oauthlib-1.0.0-1.el8.n 165 kB/s | 43 kB 00:00 2023-09-06T18:36:04.359 INFO:teuthology.orchestra.run.smithi181.stdout:(6/13): python3-pyasn1-modules-0.3.7-6.el8.noar 153 kB/s | 110 kB 00:00 2023-09-06T18:36:04.385 INFO:teuthology.orchestra.run.smithi181.stdout:(7/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 147 kB/s | 126 kB 00:00 2023-09-06T18:36:04.401 INFO:teuthology.orchestra.run.smithi181.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 99 kB/s | 33 kB 00:00 2023-09-06T18:36:04.814 INFO:teuthology.orchestra.run.smithi181.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 36 kB/s | 16 kB 00:00 2023-09-06T18:36:04.844 INFO:teuthology.orchestra.run.smithi181.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 177 kB/s | 81 kB 00:00 2023-09-06T18:36:04.961 INFO:teuthology.orchestra.run.smithi181.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 437 kB/s | 64 kB 00:00 2023-09-06T18:36:05.094 INFO:teuthology.orchestra.run.smithi181.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 245 kB/s | 61 kB 00:00 2023-09-06T18:36:05.203 INFO:teuthology.orchestra.run.smithi181.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 1.5 MB/s | 1.2 MB 00:00 2023-09-06T18:36:05.203 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:36:05.204 INFO:teuthology.orchestra.run.smithi181.stdout:Total 589 kB/s | 1.9 MB 00:03 2023-09-06T18:36:05.267 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:36:05.283 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:36:05.283 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:36:05.325 INFO:teuthology.orchestra.run.smithi099.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 1.1 MB/s | 4.8 MB 00:04 2023-09-06T18:36:05.325 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:36:05.326 INFO:teuthology.orchestra.run.smithi099.stdout:Total 1.2 MB/s | 6.0 MB 00:04 2023-09-06T18:36:05.370 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:36:05.373 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:36:05.381 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:36:05.393 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:36:05.393 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:36:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:36:05.504 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:36:05.852 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:36:05.894 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:36:06.133 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-09-06T18:36:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-09-06T18:36:06.342 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-09-06T18:36:06.577 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2023-09-06T18:36:06.840 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-09-06T18:36:07.065 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-09-06T18:36:07.205 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-09-06T18:36:07.296 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-09-06T18:36:07.604 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-09-06T18:36:07.623 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-09-06T18:36:07.880 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-09-06T18:36:07.925 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-09-06T18:36:08.159 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 5/5 2023-09-06T18:36:08.444 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 5/5 2023-09-06T18:36:08.444 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 1/5 2023-09-06T18:36:08.445 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-09-06T18:36:08.445 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-09-06T18:36:08.445 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-09-06T18:36:08.905 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-09-06T18:36:09.070 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-09-06T18:36:09.070 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:09.071 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:36:09.071 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:36:09.071 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-06T18:36:09.071 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-06T18:36:09.071 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-06T18:36:09.072 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-06T18:36:09.072 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:09.072 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:36:09.165 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-09-06T18:36:09.181 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-fuse 2023-09-06T18:36:09.399 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-09-06T18:36:09.625 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-09-06T18:36:09.783 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:18.0.0-5801.g5535a92b.el8.noarch 13/13 2023-09-06T18:36:10.020 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:29 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:36:10.075 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5801.g5535a92b.el8.noarch 13/13 2023-09-06T18:36:10.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5801.g5535a92b.el8.noarch 1/13 2023-09-06T18:36:10.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-09-06T18:36:10.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-09-06T18:36:10.076 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-09-06T18:36:10.076 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-09-06T18:36:10.076 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-09-06T18:36:10.076 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-09-06T18:36:10.077 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-09-06T18:36:10.077 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-09-06T18:36:10.077 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-09-06T18:36:10.077 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-09-06T18:36:10.077 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2023-09-06T18:36:10.761 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-09-06T18:36:10.761 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:10.761 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:36:10.761 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:36:10.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-06T18:36:10.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-06T18:36:10.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-06T18:36:10.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-06T18:36:10.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-06T18:36:10.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-06T18:36:10.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-06T18:36:10.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-06T18:36:10.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-06T18:36:10.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-06T18:36:10.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-06T18:36:10.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-06T18:36:10.764 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:10.764 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:36:10.896 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-cephadm 2023-09-06T18:36:11.561 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:11.562 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:11.562 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-06T18:36:11.562 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:11.563 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:36:11.563 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 861 k 2023-09-06T18:36:11.563 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-09-06T18:36:11.563 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-09-06T18:36:11.563 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:11.564 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:36:11.564 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:11.564 INFO:teuthology.orchestra.run.smithi099.stdout:Install 2 Packages 2023-09-06T18:36:11.564 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:11.565 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 944 k 2023-09-06T18:36:11.565 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 3.0 M 2023-09-06T18:36:11.565 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:36:11.735 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:21 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:36:12.424 INFO:teuthology.orchestra.run.smithi099.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 170 kB/s | 83 kB 00:00 2023-09-06T18:36:13.016 INFO:teuthology.orchestra.run.smithi099.stdout:(2/2): ceph-fuse-18.0.0-5801.g5535a92b.el8.x86_ 797 kB/s | 861 kB 00:01 2023-09-06T18:36:13.017 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:36:13.017 INFO:teuthology.orchestra.run.smithi099.stdout:Total 649 kB/s | 944 kB 00:01 2023-09-06T18:36:13.029 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:36:13.052 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:36:13.053 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:36:13.145 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:36:13.148 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:36:13.279 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:36:13.280 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:13.280 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-09-06T18:36:13.280 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:13.280 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:36:13.281 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 132 k 2023-09-06T18:36:13.281 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-09-06T18:36:13.281 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-06T18:36:13.281 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-09-06T18:36:13.281 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-09-06T18:36:13.282 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-06T18:36:13.282 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:13.282 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:36:13.282 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:13.283 INFO:teuthology.orchestra.run.smithi181.stdout:Install 5 Packages 2023-09-06T18:36:13.283 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:13.283 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 6.0 M 2023-09-06T18:36:13.284 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 26 M 2023-09-06T18:36:13.284 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:36:13.322 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:36:13.624 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2023-09-06T18:36:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 2/2 2023-09-06T18:36:14.061 INFO:teuthology.orchestra.run.smithi181.stdout:(1/5): ceph-mgr-cephadm-18.0.0-5801.g5535a92b.e 339 kB/s | 132 kB 00:00 2023-09-06T18:36:14.233 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 2/2 2023-09-06T18:36:14.233 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 1/2 2023-09-06T18:36:14.844 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2023-09-06T18:36:14.844 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:14.844 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:36:14.845 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2023-09-06T18:36:14.845 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:14.845 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:36:14.861 INFO:teuthology.orchestra.run.smithi181.stdout:(2/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 595 kB/s | 476 kB 00:00 2023-09-06T18:36:15.038 INFO:teuthology.orchestra.run.smithi181.stdout:(3/5): python3-natsort-7.1.1-2.el8.noarch.rpm 341 kB/s | 60 kB 00:00 2023-09-06T18:36:15.077 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-volume 2023-09-06T18:36:15.320 INFO:teuthology.orchestra.run.smithi181.stdout:(4/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 326 kB/s | 538 kB 00:01 2023-09-06T18:36:15.935 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:34 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:36:17.477 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:17.478 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-06T18:36:17.479 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:17.479 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:36:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 260 k 2023-09-06T18:36:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:17.480 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:36:17.480 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:17.480 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2023-09-06T18:36:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:17.480 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 260 k 2023-09-06T18:36:17.481 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 1.3 M 2023-09-06T18:36:17.481 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:36:17.703 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-volume-18.0.0-5801.g5535a92b.el8.noarch.rp 1.1 MB/s | 260 kB 00:00 2023-09-06T18:36:17.704 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:36:17.704 INFO:teuthology.orchestra.run.smithi099.stdout:Total 1.1 MB/s | 260 kB 00:00 2023-09-06T18:36:17.705 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:36:17.714 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:36:17.715 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:36:17.751 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:36:17.754 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:36:17.930 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:36:17.986 INFO:teuthology.orchestra.run.smithi181.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 1.1 MB/s | 4.8 MB 00:04 2023-09-06T18:36:17.988 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:36:17.988 INFO:teuthology.orchestra.run.smithi181.stdout:Total 1.3 MB/s | 6.0 MB 00:04 2023-09-06T18:36:18.043 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:36:18.055 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:36:18.056 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:36:18.167 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:36:18.170 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:36:18.542 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:36:18.870 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-09-06T18:36:19.407 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:18.0.0-5801.g5535a92b.el8.noarch 1/1 2023-09-06T18:36:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:18.0.0-5801.g5535a92b.el8.noarch 1/1 2023-09-06T18:36:19.852 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-09-06T18:36:20.243 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-09-06T18:36:20.462 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:18.0.0-5801.g5535a92b.el8.noarch 1/1 2023-09-06T18:36:20.462 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:20.462 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:36:20.463 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:36:20.463 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:20.463 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:36:20.513 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-09-06T18:36:20.706 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 5/5 2023-09-06T18:36:20.711 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install librados-devel 2023-09-06T18:36:20.989 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 5/5 2023-09-06T18:36:20.990 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 1/5 2023-09-06T18:36:20.990 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-09-06T18:36:20.990 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-09-06T18:36:20.990 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-09-06T18:36:21.567 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:40 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:36:21.594 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-09-06T18:36:21.595 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:21.595 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:36:21.595 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:36:21.596 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-06T18:36:21.596 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-06T18:36:21.597 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-06T18:36:21.597 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-06T18:36:21.597 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:21.598 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:36:21.714 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-fuse 2023-09-06T18:36:22.557 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:32 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:36:23.119 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:23.120 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:23.121 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repo Size 2023-09-06T18:36:23.121 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:23.121 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:36:23.121 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 139 k 2023-09-06T18:36:23.121 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:23.122 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:36:23.122 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:23.122 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2023-09-06T18:36:23.122 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:23.122 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 139 k 2023-09-06T18:36:23.123 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 513 k 2023-09-06T18:36:23.123 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:36:23.309 INFO:teuthology.orchestra.run.smithi099.stdout:librados-devel-18.0.0-5801.g5535a92b.el8.x86_64 740 kB/s | 139 kB 00:00 2023-09-06T18:36:23.310 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:36:23.310 INFO:teuthology.orchestra.run.smithi099.stdout:Total 735 kB/s | 139 kB 00:00 2023-09-06T18:36:23.310 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:36:23.315 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:36:23.315 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:36:23.365 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:36:23.370 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:36:23.485 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:36:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:24.069 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:24.116 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:36:24.117 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:24.117 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-09-06T18:36:24.117 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:24.117 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:36:24.118 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 861 k 2023-09-06T18:36:24.118 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-09-06T18:36:24.118 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-09-06T18:36:24.118 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:24.118 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:36:24.119 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:24.119 INFO:teuthology.orchestra.run.smithi181.stdout:Install 2 Packages 2023-09-06T18:36:24.119 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:24.120 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 944 k 2023-09-06T18:36:24.120 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 3.0 M 2023-09-06T18:36:24.120 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:36:24.585 INFO:teuthology.orchestra.run.smithi181.stdout:(1/2): ceph-fuse-18.0.0-5801.g5535a92b.el8.x86_ 3.1 MB/s | 861 kB 00:00 2023-09-06T18:36:24.677 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:24.677 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:24.677 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:36:24.678 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:36:24.678 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:24.678 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:36:24.902 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install libcephfs2 2023-09-06T18:36:25.760 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:44 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:36:26.849 INFO:teuthology.orchestra.run.smithi181.stdout:(2/2): fuse-2.9.7-17.el8.x86_64.rpm 33 kB/s | 83 kB 00:02 2023-09-06T18:36:26.850 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:36:26.850 INFO:teuthology.orchestra.run.smithi181.stdout:Total 346 kB/s | 944 kB 00:02 2023-09-06T18:36:26.860 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:36:26.883 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:36:26.884 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:36:26.976 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:36:26.979 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:36:27.152 INFO:teuthology.orchestra.run.smithi099.stdout:Package libcephfs2-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:36:27.153 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:36:27.330 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:27.331 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2023-09-06T18:36:27.332 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:36:27.421 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2023-09-06T18:36:27.424 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install libcephfs-devel 2023-09-06T18:36:27.631 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 2/2 2023-09-06T18:36:28.039 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 2/2 2023-09-06T18:36:28.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 1/2 2023-09-06T18:36:28.272 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:47 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:36:28.690 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2023-09-06T18:36:28.690 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:28.690 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:36:28.690 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2023-09-06T18:36:28.691 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:28.691 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:36:28.954 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-volume 2023-09-06T18:36:29.816 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:39 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:36:29.828 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:29.829 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:29.829 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repo Size 2023-09-06T18:36:29.829 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:29.830 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:36:29.830 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 31 k 2023-09-06T18:36:29.830 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:29.830 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:36:29.830 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:29.831 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2023-09-06T18:36:29.831 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:29.831 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 31 k 2023-09-06T18:36:29.832 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 140 k 2023-09-06T18:36:29.832 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:36:30.000 INFO:teuthology.orchestra.run.smithi099.stdout:libcephfs-devel-18.0.0-5801.g5535a92b.el8.x86_6 183 kB/s | 31 kB 00:00 2023-09-06T18:36:30.001 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:36:30.001 INFO:teuthology.orchestra.run.smithi099.stdout:Total 182 kB/s | 31 kB 00:00 2023-09-06T18:36:30.001 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:36:30.004 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:36:30.005 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:36:30.035 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:36:30.038 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:36:30.100 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:36:30.704 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:31.011 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:31.374 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:36:31.375 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:31.376 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-09-06T18:36:31.376 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:31.376 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:36:31.376 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:18.0.0-5801.g5535a92b.el8 ceph-noarch 260 k 2023-09-06T18:36:31.377 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:31.377 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:36:31.377 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:31.377 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2023-09-06T18:36:31.377 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:31.377 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 260 k 2023-09-06T18:36:31.378 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 1.3 M 2023-09-06T18:36:31.378 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:36:31.602 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:31.602 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:31.603 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:36:31.603 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:36:31.603 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:31.603 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:36:31.719 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install librados2 2023-09-06T18:36:31.944 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-volume-18.0.0-5801.g5535a92b.el8.noarch.rp 459 kB/s | 260 kB 00:00 2023-09-06T18:36:31.944 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:36:31.945 INFO:teuthology.orchestra.run.smithi181.stdout:Total 458 kB/s | 260 kB 00:00 2023-09-06T18:36:31.945 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:36:31.957 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:36:31.957 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:36:31.992 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:36:31.995 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:36:32.171 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:36:32.447 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:18.0.0-5801.g5535a92b.el8.noarch 1/1 2023-09-06T18:36:32.577 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:51 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:36:32.868 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:18.0.0-5801.g5535a92b.el8.noarch 1/1 2023-09-06T18:36:33.474 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:18.0.0-5801.g5535a92b.el8.noarch 1/1 2023-09-06T18:36:33.474 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:33.475 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:36:33.475 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:18.0.0-5801.g5535a92b.el8.noarch 2023-09-06T18:36:33.475 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:33.475 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:36:33.674 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install librados-devel 2023-09-06T18:36:33.946 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:36:34.121 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:34.122 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2023-09-06T18:36:34.122 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:36:34.206 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install librbd1 2023-09-06T18:36:34.533 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:44 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:36:35.098 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:54 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:36:36.097 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:36:36.098 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:36.098 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repo Size 2023-09-06T18:36:36.099 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:36.099 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:36:36.099 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 139 k 2023-09-06T18:36:36.099 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:36.099 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:36:36.100 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:36.100 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2023-09-06T18:36:36.100 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:36.100 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 139 k 2023-09-06T18:36:36.101 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 513 k 2023-09-06T18:36:36.101 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:36:36.290 INFO:teuthology.orchestra.run.smithi181.stdout:librados-devel-18.0.0-5801.g5535a92b.el8.x86_64 729 kB/s | 139 kB 00:00 2023-09-06T18:36:36.290 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:36:36.291 INFO:teuthology.orchestra.run.smithi181.stdout:Total 724 kB/s | 139 kB 00:00 2023-09-06T18:36:36.291 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:36:36.296 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:36:36.296 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:36:36.346 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:36:36.349 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:36:36.467 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:36:36.484 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:36:36.660 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:36.661 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2023-09-06T18:36:36.662 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:36:36.752 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install python3-rados 2023-09-06T18:36:36.949 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:37.316 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:37.615 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:56 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:36:37.939 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:37.939 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:37.939 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:36:37.940 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:36:37.940 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:37.940 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:36:38.183 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install libcephfs2 2023-09-06T18:36:38.991 INFO:teuthology.orchestra.run.smithi099.stdout:Package python3-rados-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:36:39.041 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:49 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:36:39.167 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:39.168 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2023-09-06T18:36:39.168 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:36:39.270 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install python3-rgw 2023-09-06T18:36:40.135 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:59 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:36:40.417 INFO:teuthology.orchestra.run.smithi181.stdout:Package libcephfs2-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:36:40.595 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:36:40.596 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2023-09-06T18:36:40.596 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:36:40.706 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install libcephfs-devel 2023-09-06T18:36:41.487 INFO:teuthology.orchestra.run.smithi099.stdout:Package python3-rgw-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:36:41.555 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:51 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:36:41.670 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:41.671 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2023-09-06T18:36:41.671 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:36:41.769 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install python3-cephfs 2023-09-06T18:36:42.631 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:03:01 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:36:43.123 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:36:43.124 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:43.125 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repo Size 2023-09-06T18:36:43.125 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:43.125 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:36:43.125 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 31 k 2023-09-06T18:36:43.125 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:43.126 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:36:43.126 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:36:43.126 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2023-09-06T18:36:43.126 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:43.126 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 31 k 2023-09-06T18:36:43.127 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 140 k 2023-09-06T18:36:43.127 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:36:43.271 INFO:teuthology.orchestra.run.smithi181.stdout:libcephfs-devel-18.0.0-5801.g5535a92b.el8.x86_6 213 kB/s | 31 kB 00:00 2023-09-06T18:36:43.272 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:36:43.272 INFO:teuthology.orchestra.run.smithi181.stdout:Total 212 kB/s | 31 kB 00:00 2023-09-06T18:36:43.272 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:36:43.275 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:36:43.275 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:36:43.303 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:36:43.306 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:36:43.365 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:36:43.602 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:43.869 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:44.014 INFO:teuthology.orchestra.run.smithi099.stdout:Package python3-cephfs-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:36:44.190 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:44.192 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2023-09-06T18:36:44.192 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:36:44.327 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install python3-rbd 2023-09-06T18:36:44.434 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:44.434 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:44.434 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:36:44.435 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:36:44.435 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:36:44.435 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:36:44.547 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install librados2 2023-09-06T18:36:45.172 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:03:04 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:36:45.398 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:55 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:36:46.554 INFO:teuthology.orchestra.run.smithi099.stdout:Package python3-rbd-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:36:46.733 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:46.734 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2023-09-06T18:36:46.734 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:36:46.769 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:36:46.831 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install rbd-fuse 2023-09-06T18:36:46.948 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:36:46.949 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2023-09-06T18:36:46.949 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:36:47.057 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install librbd1 2023-09-06T18:36:47.686 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:03:06 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:36:47.907 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:57 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:36:49.247 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:49.248 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:49.249 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2023-09-06T18:36:49.249 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:49.249 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:36:49.249 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 93 k 2023-09-06T18:36:49.250 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:49.250 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:36:49.250 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:49.250 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2023-09-06T18:36:49.250 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:49.251 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 93 k 2023-09-06T18:36:49.251 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 276 k 2023-09-06T18:36:49.251 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:36:49.285 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:36:49.441 INFO:teuthology.orchestra.run.smithi099.stdout:rbd-fuse-18.0.0-5801.g5535a92b.el8.x86_64.rpm 487 kB/s | 93 kB 00:00 2023-09-06T18:36:49.442 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:36:49.442 INFO:teuthology.orchestra.run.smithi099.stdout:Total 484 kB/s | 93 kB 00:00 2023-09-06T18:36:49.443 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:36:49.452 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:36:49.453 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:36:49.461 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:36:49.462 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2023-09-06T18:36:49.462 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:36:49.502 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:36:49.505 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:36:49.570 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install python3-rados 2023-09-06T18:36:49.605 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:36:49.815 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:50.099 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:50.422 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:03:00 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:36:50.718 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:50.718 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:50.718 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:36:50.718 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:36:50.719 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:50.719 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:36:50.954 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install rbd-mirror 2023-09-06T18:36:51.786 INFO:teuthology.orchestra.run.smithi181.stdout:Package python3-rados-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:36:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:03:10 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:36:51.965 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:36:51.966 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2023-09-06T18:36:51.967 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:36:52.076 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install python3-rgw 2023-09-06T18:36:52.927 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:03:02 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:36:53.376 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:53.377 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:53.377 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repo Size 2023-09-06T18:36:53.377 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:53.377 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:36:53.377 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 3.5 M 2023-09-06T18:36:53.378 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:53.378 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:36:53.378 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:53.378 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2023-09-06T18:36:53.379 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:53.379 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 3.5 M 2023-09-06T18:36:53.379 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 17 M 2023-09-06T18:36:53.379 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:36:54.127 INFO:teuthology.orchestra.run.smithi099.stdout:rbd-mirror-18.0.0-5801.g5535a92b.el8.x86_64.rpm 4.7 MB/s | 3.5 MB 00:00 2023-09-06T18:36:54.127 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:36:54.128 INFO:teuthology.orchestra.run.smithi099.stdout:Total 4.7 MB/s | 3.5 MB 00:00 2023-09-06T18:36:54.128 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:36:54.141 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:36:54.141 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:36:54.203 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:36:54.206 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:36:54.322 INFO:teuthology.orchestra.run.smithi181.stdout:Package python3-rgw-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:36:54.499 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:36:54.501 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2023-09-06T18:36:54.501 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:36:54.597 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install python3-cephfs 2023-09-06T18:36:54.654 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:36:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:55.268 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:55.444 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:03:05 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:36:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:55.862 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:36:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:36:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:55.862 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:36:56.033 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install rbd-nbd 2023-09-06T18:36:56.818 INFO:teuthology.orchestra.run.smithi181.stdout:Package python3-cephfs-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:36:56.888 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:03:15 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:36:56.993 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:36:56.994 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2023-09-06T18:36:56.994 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:36:57.093 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install python3-rbd 2023-09-06T18:36:57.948 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:03:07 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:36:58.450 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:36:58.451 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:58.452 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2023-09-06T18:36:58.452 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:58.452 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:36:58.452 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 179 k 2023-09-06T18:36:58.452 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:58.453 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:36:58.453 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:36:58.453 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2023-09-06T18:36:58.453 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:36:58.453 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 179 k 2023-09-06T18:36:58.454 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 565 k 2023-09-06T18:36:58.454 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:36:58.763 INFO:teuthology.orchestra.run.smithi099.stdout:rbd-nbd-18.0.0-5801.g5535a92b.el8.x86_64.rpm 575 kB/s | 179 kB 00:00 2023-09-06T18:36:58.764 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:36:58.764 INFO:teuthology.orchestra.run.smithi099.stdout:Total 573 kB/s | 179 kB 00:00 2023-09-06T18:36:58.764 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:36:58.776 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:36:58.776 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:36:58.826 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:36:58.829 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:36:58.947 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:36:59.163 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:59.328 INFO:teuthology.orchestra.run.smithi181.stdout:Package python3-rbd-2:18.0.0-5801.g5535a92b.el8.x86_64 is already installed. 2023-09-06T18:36:59.465 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-nbd-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:36:59.504 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:36:59.506 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2023-09-06T18:36:59.506 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:36:59.612 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install rbd-fuse 2023-09-06T18:37:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:37:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:00.098 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:37:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:37:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:00.099 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:37:00.331 DEBUG:teuthology.parallel:result is None 2023-09-06T18:37:00.462 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:03:10 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:37:02.014 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:37:02.015 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:37:02.016 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2023-09-06T18:37:02.016 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:37:02.016 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:37:02.016 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 93 k 2023-09-06T18:37:02.016 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:02.017 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:37:02.017 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:37:02.017 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2023-09-06T18:37:02.017 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:02.018 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 93 k 2023-09-06T18:37:02.018 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 276 k 2023-09-06T18:37:02.018 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:37:02.525 INFO:teuthology.orchestra.run.smithi181.stdout:rbd-fuse-18.0.0-5801.g5535a92b.el8.x86_64.rpm 184 kB/s | 93 kB 00:00 2023-09-06T18:37:02.572 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:37:02.573 INFO:teuthology.orchestra.run.smithi181.stdout:Total 183 kB/s | 93 kB 00:00 2023-09-06T18:37:02.573 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:37:02.574 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:37:02.574 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:37:02.584 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:37:02.587 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:37:02.696 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:37:02.907 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:37:03.194 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:37:03.805 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:37:03.805 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:03.805 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:37:03.806 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:37:03.806 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:03.806 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:37:04.037 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install rbd-mirror 2023-09-06T18:37:04.901 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:03:14 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:37:06.449 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:37:06.450 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:37:06.451 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repo Size 2023-09-06T18:37:06.451 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:37:06.451 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:37:06.451 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 3.5 M 2023-09-06T18:37:06.451 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:06.452 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:37:06.452 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:37:06.452 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2023-09-06T18:37:06.452 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:06.452 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 3.5 M 2023-09-06T18:37:06.453 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 17 M 2023-09-06T18:37:06.453 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:37:07.169 INFO:teuthology.orchestra.run.smithi181.stdout:rbd-mirror-18.0.0-5801.g5535a92b.el8.x86_64.rpm 4.9 MB/s | 3.5 MB 00:00 2023-09-06T18:37:07.170 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:37:07.170 INFO:teuthology.orchestra.run.smithi181.stdout:Total 4.9 MB/s | 3.5 MB 00:00 2023-09-06T18:37:07.170 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:37:07.182 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:37:07.182 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:37:07.244 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:37:07.247 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:37:07.694 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:37:07.899 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:37:08.305 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:37:08.930 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:37:08.931 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:08.931 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:37:08.931 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:37:08.931 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:08.932 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:37:09.347 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install rbd-nbd 2023-09-06T18:37:10.205 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:03:20 ago on Wed 06 Sep 2023 06:33:50 PM UTC. 2023-09-06T18:37:11.752 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-09-06T18:37:11.753 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:37:11.753 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2023-09-06T18:37:11.753 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:37:11.753 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-09-06T18:37:11.754 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:18.0.0-5801.g5535a92b.el8 ceph 179 k 2023-09-06T18:37:11.754 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:11.754 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-09-06T18:37:11.754 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-09-06T18:37:11.754 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2023-09-06T18:37:11.755 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:11.755 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 179 k 2023-09-06T18:37:11.755 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 565 k 2023-09-06T18:37:11.755 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-09-06T18:37:12.155 INFO:teuthology.orchestra.run.smithi181.stdout:rbd-nbd-18.0.0-5801.g5535a92b.el8.x86_64.rpm 445 kB/s | 179 kB 00:00 2023-09-06T18:37:12.156 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:37:12.156 INFO:teuthology.orchestra.run.smithi181.stdout:Total 444 kB/s | 179 kB 00:00 2023-09-06T18:37:12.157 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-09-06T18:37:12.167 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-09-06T18:37:12.168 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-09-06T18:37:12.217 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-09-06T18:37:12.220 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-09-06T18:37:12.339 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-09-06T18:37:12.581 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:37:12.869 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-nbd-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:37:13.482 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:18.0.0-5801.g5535a92b.el8.x86_64 1/1 2023-09-06T18:37:13.482 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:13.483 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-09-06T18:37:13.483 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:18.0.0-5801.g5535a92b.el8.x86_64 2023-09-06T18:37:13.483 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:13.484 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-09-06T18:37:13.717 DEBUG:teuthology.parallel:result is None 2023-09-06T18:37:13.718 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5535a92bd618b7e580a2fce32e9dfab6620d1086 2023-09-06T18:37:13.997 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-06T18:37:14.025 INFO:teuthology.orchestra.run.smithi099.stdout:18.0.0-5801.g5535a92b.el8 2023-09-06T18:37:14.026 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5801.g5535a92b.el8 2023-09-06T18:37:14.026 INFO:teuthology.task.install:The correct ceph version 18.0.0-5801.g5535a92b is installed. 2023-09-06T18:37:14.027 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5535a92bd618b7e580a2fce32e9dfab6620d1086 2023-09-06T18:37:14.166 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-06T18:37:14.189 INFO:teuthology.orchestra.run.smithi181.stdout:18.0.0-5801.g5535a92b.el8 2023-09-06T18:37:14.190 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5801.g5535a92b.el8 2023-09-06T18:37:14.190 INFO:teuthology.task.install:The correct ceph version 18.0.0-5801.g5535a92b is installed. 2023-09-06T18:37:14.191 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-06T18:37:14.191 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:14.192 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-06T18:37:14.226 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:14.226 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-06T18:37:14.258 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-06T18:37:14.259 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:14.259 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2023-09-06T18:37:14.297 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-06T18:37:14.368 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:14.368 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2023-09-06T18:37:14.395 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-06T18:37:14.463 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-06T18:37:14.463 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:14.464 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-06T18:37:14.493 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-06T18:37:14.562 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:14.563 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-06T18:37:14.590 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-06T18:37:14.657 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-06T18:37:14.657 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:14.658 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2023-09-06T18:37:14.686 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-06T18:37:14.754 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:14.755 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2023-09-06T18:37:14.782 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-06T18:37:14.849 INFO:teuthology.run_tasks:Running task ceph... 2023-09-06T18:37:14.925 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-09-06T18:37:14.927 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /var/log/ceph 2023-09-06T18:37:14.929 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /var/log/ceph 2023-09-06T18:37:14.956 INFO:tasks.ceph:Disabling ceph logrotate... 2023-09-06T18:37:14.956 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-06T18:37:14.997 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-06T18:37:15.024 INFO:tasks.ceph:Creating extra log directories... 2023-09-06T18:37:15.024 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-06T18:37:15.065 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-06T18:37:15.102 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-09-06T18:37:15.102 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-09-06T18:37:15.103 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243', 'branch': 'wip-eric-testing-1', 'description': 'rgw/notifications/{beast bluestore-bitmap fixed-2 ignore-pg-availability overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}}', 'email': 'ivancich@redhat.com', 'first_in_suite': False, 'job_id': '7390243', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-eric-testing-1'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': '5535a92bd618b7e580a2fce32e9dfab6620d1086'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '5535a92bd618b7e580a2fce32e9dfab6620d1086'}}, 'rgw': {'frontend': 'beast', 'storage classes': 'LUKEWARM, FROZEN'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-eric-testing-1', 'sha1': '5535a92bd618b7e580a2fce32e9dfab6620d1086'}}, 'owner': 'scheduled_ivancich@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 831, 'sha1': '5535a92bd618b7e580a2fce32e9dfab6620d1086', 'sleep_before_teardown': 0, 'subset': '1/50', 'suite': 'rgw', 'suite_branch': 'wip-eric-testing-1', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '5535a92bd618b7e580a2fce32e9dfab6620d1086', 'targets': {'smithi099.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGgXCd8FtdPyuvBoDz1Odlgd52mKpJX9T2D8O3a4AwUgUBoffGlDGApGL+FaQmtbp7NgosmSVVFT864Bf5rUyyM=', 'smithi181.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBxZTiiM0jxT4QB+tCztuafD3KDXgRBguwmwjNkdFGh/L+PidvLrqhDkcqEGy+D7vC5x6bcz2eMfr5qOlmg9rC4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': None}}, {'rabbitmq': {'client.0': None}}, {'notification-tests': {'client.0': {'extra_attr': ['amqp_test'], 'rgw_server': 'client.0'}}}, {'kafka': {'client.0': {'kafka_version': '2.6.0'}}}, {'notification-tests': {'client.0': {'extra_attr': ['kafka_test'], 'rgw_server': 'client.0'}}}, {'notification-tests': {'client.0': {'rgw_server': 'client.0'}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-09-06_18:12:39', 'tube': 'smithi', 'user': 'ivancich', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1394'} 2023-09-06T18:37:15.103 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-06T18:37:15.133 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-06T18:37:15.152 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/run/ceph 2023-09-06T18:37:15.191 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/run/ceph 2023-09-06T18:37:15.219 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:15.220 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2023-09-06T18:37:15.274 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-06T18:37:15.274 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2023-09-06T18:37:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-06T18:37:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-06T18:37:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 388913 Links: 1 2023-09-06T18:37:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-06T18:37:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-09-06T18:37:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-09-06 18:36:55.021508723 +0000 2023-09-06T18:37:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-09-06 18:32:15.490819795 +0000 2023-09-06T18:37:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-09-06 18:32:15.490819795 +0000 2023-09-06T18:37:15.335 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-09-06T18:37:15.335 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-06T18:37:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-09-06T18:37:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-09-06T18:37:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000206507 s, 2.5 MB/s 2023-09-06T18:37:15.401 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-06T18:37:15.461 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2023-09-06T18:37:15.519 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-06T18:37:15.520 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-06T18:37:15.520 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 394415 Links: 1 2023-09-06T18:37:15.520 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-06T18:37:15.520 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-09-06T18:37:15.521 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-09-06 18:36:55.021508723 +0000 2023-09-06T18:37:15.521 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-09-06 18:32:16.001808139 +0000 2023-09-06T18:37:15.521 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-09-06 18:32:16.001808139 +0000 2023-09-06T18:37:15.521 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-09-06T18:37:15.522 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-06T18:37:15.585 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-09-06T18:37:15.585 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-09-06T18:37:15.586 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000218609 s, 2.3 MB/s 2023-09-06T18:37:15.587 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-06T18:37:15.645 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2023-09-06T18:37:15.703 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-06T18:37:15.704 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-06T18:37:15.704 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 395472 Links: 1 2023-09-06T18:37:15.704 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-06T18:37:15.704 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-09-06T18:37:15.704 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-09-06 18:36:55.022508701 +0000 2023-09-06T18:37:15.705 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-09-06 18:32:16.531796049 +0000 2023-09-06T18:37:15.705 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-09-06 18:32:16.531796049 +0000 2023-09-06T18:37:15.705 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-09-06T18:37:15.705 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-06T18:37:15.769 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-09-06T18:37:15.769 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-09-06T18:37:15.769 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000213889 s, 2.4 MB/s 2023-09-06T18:37:15.771 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-06T18:37:15.831 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2023-09-06T18:37:15.889 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-06T18:37:15.890 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-06T18:37:15.890 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 391840 Links: 1 2023-09-06T18:37:15.891 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-06T18:37:15.891 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-09-06T18:37:15.891 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-09-06 18:36:55.022508701 +0000 2023-09-06T18:37:15.892 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-09-06 18:32:17.055784096 +0000 2023-09-06T18:37:15.892 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-09-06 18:32:17.055784096 +0000 2023-09-06T18:37:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-09-06T18:37:15.892 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-06T18:37:15.955 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-09-06T18:37:15.956 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-09-06T18:37:15.956 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000252622 s, 2.0 MB/s 2023-09-06T18:37:15.957 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-06T18:37:16.016 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-06T18:37:16.017 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:16.017 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2023-09-06T18:37:16.037 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-06T18:37:16.037 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2023-09-06T18:37:16.096 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-06T18:37:16.096 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-06T18:37:16.096 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 382767 Links: 1 2023-09-06T18:37:16.096 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-06T18:37:16.097 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-09-06T18:37:16.097 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-09-06 18:37:08.054593774 +0000 2023-09-06T18:37:16.097 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-09-06 18:32:04.984825773 +0000 2023-09-06T18:37:16.097 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-09-06 18:32:04.984825773 +0000 2023-09-06T18:37:16.097 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-09-06T18:37:16.098 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-06T18:37:16.162 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-09-06T18:37:16.163 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-09-06T18:37:16.163 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000202009 s, 2.5 MB/s 2023-09-06T18:37:16.166 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-06T18:37:16.226 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2023-09-06T18:37:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-06T18:37:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-06T18:37:16.285 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 394356 Links: 1 2023-09-06T18:37:16.285 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-06T18:37:16.286 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-09-06T18:37:16.286 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-09-06 18:37:08.054593774 +0000 2023-09-06T18:37:16.286 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-09-06 18:32:05.512815204 +0000 2023-09-06T18:37:16.286 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-09-06 18:32:05.512815204 +0000 2023-09-06T18:37:16.286 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-09-06T18:37:16.287 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-06T18:37:16.351 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-09-06T18:37:16.351 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-09-06T18:37:16.351 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000213057 s, 2.4 MB/s 2023-09-06T18:37:16.353 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-06T18:37:16.413 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2023-09-06T18:37:16.472 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-06T18:37:16.472 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-06T18:37:16.472 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 396307 Links: 1 2023-09-06T18:37:16.472 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-06T18:37:16.473 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-09-06T18:37:16.473 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-09-06 18:37:08.054593774 +0000 2023-09-06T18:37:16.473 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-09-06 18:32:06.024804955 +0000 2023-09-06T18:37:16.473 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-09-06 18:32:06.024804955 +0000 2023-09-06T18:37:16.473 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-09-06T18:37:16.474 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-06T18:37:16.538 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-09-06T18:37:16.538 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-09-06T18:37:16.538 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000210094 s, 2.4 MB/s 2023-09-06T18:37:16.539 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-06T18:37:16.598 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2023-09-06T18:37:16.657 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-06T18:37:16.657 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-06T18:37:16.658 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 394207 Links: 1 2023-09-06T18:37:16.658 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-06T18:37:16.658 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-09-06T18:37:16.658 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-09-06 18:37:08.054593774 +0000 2023-09-06T18:37:16.658 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-09-06 18:32:06.535794726 +0000 2023-09-06T18:37:16.659 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-09-06 18:32:06.535794726 +0000 2023-09-06T18:37:16.659 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-09-06T18:37:16.659 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-06T18:37:16.723 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-09-06T18:37:16.723 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-09-06T18:37:16.723 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000246911 s, 2.1 MB/s 2023-09-06T18:37:16.724 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-06T18:37:16.783 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-06T18:37:16.784 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-09-06T18:37:16.784 INFO:tasks.ceph:Generating config... 2023-09-06T18:37:16.785 INFO:tasks.ceph:[client] debug rgw = 20 2023-09-06T18:37:16.786 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2023-09-06T18:37:16.786 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2023-09-06T18:37:16.786 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2023-09-06T18:37:16.787 INFO:tasks.ceph:[client] rgw lc debug interval = 10 2023-09-06T18:37:16.787 INFO:tasks.ceph:[client] setgroup = ceph 2023-09-06T18:37:16.787 INFO:tasks.ceph:[client] setuser = ceph 2023-09-06T18:37:16.787 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2023-09-06T18:37:16.788 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2023-09-06T18:37:16.788 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-09-06T18:37:16.788 INFO:tasks.ceph:[mgr] debug ms = 1 2023-09-06T18:37:16.788 INFO:tasks.ceph:[mon] debug mon = 20 2023-09-06T18:37:16.789 INFO:tasks.ceph:[mon] debug ms = 1 2023-09-06T18:37:16.789 INFO:tasks.ceph:[mon] debug paxos = 20 2023-09-06T18:37:16.789 INFO:tasks.ceph:[osd] bdev async discard = True 2023-09-06T18:37:16.790 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-09-06T18:37:16.790 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-09-06T18:37:16.790 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-09-06T18:37:16.791 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-09-06T18:37:16.791 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-09-06T18:37:16.791 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-09-06T18:37:16.791 INFO:tasks.ceph:[osd] debug ms = 1 2023-09-06T18:37:16.792 INFO:tasks.ceph:[osd] debug osd = 20 2023-09-06T18:37:16.792 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-09-06T18:37:16.792 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-09-06T18:37:16.793 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-09-06T18:37:16.793 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-09-06T18:37:16.793 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-09-06T18:37:16.793 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-09-06T18:37:16.794 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-09-06T18:37:16.794 INFO:tasks.ceph:Setting up mon.a... 2023-09-06T18:37:16.794 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-09-06T18:37:16.860 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.keyring 2023-09-06T18:37:16.863 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-09-06T18:37:16.913 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-06T18:37:16.942 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.99'), ('mon.c', '[v2:172.21.15.99:3301,v1:172.21.15.99:6790]'), ('mon.b', '172.21.15.181')] 2023-09-06T18:37:16.942 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.99,[v2:172.21.15.99:3301,v1:172.21.15.99:6790],172.21.15.181', 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-09-06T18:37:16.943 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:16.944 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-09-06T18:37:17.000 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.99 --addv c '[v2:172.21.15.99:3301,v1:172.21.15.99:6790]' --add b 172.21.15.181 --print /home/ubuntu/cephtest/ceph.monmap 2023-09-06T18:37:17.089 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setuser ceph since I am not root 2023-09-06T18:37:17.089 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setgroup ceph since I am not root 2023-09-06T18:37:17.090 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-09-06T18:37:17.090 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: generated fsid b0ee29a8-5d11-45f6-ae33-fae8cc67c8f5 2023-09-06T18:37:17.090 INFO:teuthology.orchestra.run.smithi099.stdout:setting min_mon_release = pacific 2023-09-06T18:37:17.091 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 0 2023-09-06T18:37:17.091 INFO:teuthology.orchestra.run.smithi099.stdout:fsid b0ee29a8-5d11-45f6-ae33-fae8cc67c8f5 2023-09-06T18:37:17.091 INFO:teuthology.orchestra.run.smithi099.stdout:last_changed 2023-09-06T18:37:17.089807+0000 2023-09-06T18:37:17.091 INFO:teuthology.orchestra.run.smithi099.stdout:created 2023-09-06T18:37:17.089807+0000 2023-09-06T18:37:17.091 INFO:teuthology.orchestra.run.smithi099.stdout:min_mon_release 16 (pacific) 2023-09-06T18:37:17.092 INFO:teuthology.orchestra.run.smithi099.stdout:election_strategy: 1 2023-09-06T18:37:17.092 INFO:teuthology.orchestra.run.smithi099.stdout:0: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.a 2023-09-06T18:37:17.092 INFO:teuthology.orchestra.run.smithi099.stdout:1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2023-09-06T18:37:17.092 INFO:teuthology.orchestra.run.smithi099.stdout:2: [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] mon.c 2023-09-06T18:37:17.092 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-09-06T18:37:17.094 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-09-06T18:37:17.111 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b0ee29a8-5d11-45f6-ae33-fae8cc67c8f5... 2023-09-06T18:37:17.112 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-06T18:37:17.154 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-06T18:37:17.196 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2023-09-06T18:37:17.196 INFO:teuthology.orchestra.run.smithi099.stdout: chdir = "" 2023-09-06T18:37:17.197 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-06T18:37:17.197 INFO:teuthology.orchestra.run.smithi099.stdout: auth supported = cephx 2023-09-06T18:37:17.197 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.197 INFO:teuthology.orchestra.run.smithi099.stdout: filestore xattr use omap = true 2023-09-06T18:37:17.198 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.198 INFO:teuthology.orchestra.run.smithi099.stdout: mon clock drift allowed = 1.000 2023-09-06T18:37:17.198 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.198 INFO:teuthology.orchestra.run.smithi099.stdout: osd crush chooseleaf type = 0 2023-09-06T18:37:17.198 INFO:teuthology.orchestra.run.smithi099.stdout: auth debug = true 2023-09-06T18:37:17.199 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.199 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on old message = true 2023-09-06T18:37:17.199 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on bug = true 2023-09-06T18:37:17.199 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.199 INFO:teuthology.orchestra.run.smithi099.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-06T18:37:17.199 INFO:teuthology.orchestra.run.smithi099.stdout: mon pg warn max object skew = 0 2023-09-06T18:37:17.200 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.200 INFO:teuthology.orchestra.run.smithi099.stdout: # disable pg_autoscaler by default for new pools 2023-09-06T18:37:17.200 INFO:teuthology.orchestra.run.smithi099.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-06T18:37:17.200 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.200 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default size = 2 2023-09-06T18:37:17.201 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.201 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow primary affinity = true 2023-09-06T18:37:17.201 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow pg remap = true 2023-09-06T18:37:17.201 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on legacy crush tunables = false 2023-09-06T18:37:17.201 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on crush straw calc version zero = false 2023-09-06T18:37:17.202 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on no sortbitwise = false 2023-09-06T18:37:17.202 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on osd down out interval zero = false 2023-09-06T18:37:17.202 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on too few osds = false 2023-09-06T18:37:17.202 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-06T18:37:17.202 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-06T18:37:17.202 INFO:teuthology.orchestra.run.smithi099.stdout: mon_allow_pool_size_one = true 2023-09-06T18:37:17.203 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.203 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-06T18:37:17.203 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.203 INFO:teuthology.orchestra.run.smithi099.stdout: osd default data pool replay window = 5 2023-09-06T18:37:17.203 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.204 INFO:teuthology.orchestra.run.smithi099.stdout: mon allow pool delete = true 2023-09-06T18:37:17.204 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.204 INFO:teuthology.orchestra.run.smithi099.stdout: mon cluster log file level = debug 2023-09-06T18:37:17.204 INFO:teuthology.orchestra.run.smithi099.stdout: debug asserts on shutdown = true 2023-09-06T18:37:17.204 INFO:teuthology.orchestra.run.smithi099.stdout: mon health detail to clog = false 2023-09-06T18:37:17.205 INFO:teuthology.orchestra.run.smithi099.stdout: mon host = "172.21.15.99,[v2:172.21.15.99:3301,v1:172.21.15.99:6790],172.21.15.181" 2023-09-06T18:37:17.205 INFO:teuthology.orchestra.run.smithi099.stdout: osd_max_pg_log_entries = 10 2023-09-06T18:37:17.205 INFO:teuthology.orchestra.run.smithi099.stdout: osd_min_pg_log_entries = 10 2023-09-06T18:37:17.205 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = b0ee29a8-5d11-45f6-ae33-fae8cc67c8f5 2023-09-06T18:37:17.205 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.206 INFO:teuthology.orchestra.run.smithi099.stdout:[osd] 2023-09-06T18:37:17.206 INFO:teuthology.orchestra.run.smithi099.stdout: osd journal size = 100 2023-09-06T18:37:17.206 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.206 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub load threshold = 5.0 2023-09-06T18:37:17.206 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub max interval = 600 2023-09-06T18:37:17.207 INFO:teuthology.orchestra.run.smithi099.stdout: osd mclock profile = high_recovery_ops 2023-09-06T18:37:17.207 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.207 INFO:teuthology.orchestra.run.smithi099.stdout: osd recover clone overlap = true 2023-09-06T18:37:17.207 INFO:teuthology.orchestra.run.smithi099.stdout: osd recovery max chunk = 1048576 2023-09-06T18:37:17.207 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.208 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug shutdown = true 2023-09-06T18:37:17.208 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug op order = true 2023-09-06T18:37:17.208 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug verify stray on activate = true 2023-09-06T18:37:17.208 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.208 INFO:teuthology.orchestra.run.smithi099.stdout: osd open classes on start = true 2023-09-06T18:37:17.208 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug pg log writeout = true 2023-09-06T18:37:17.209 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.209 INFO:teuthology.orchestra.run.smithi099.stdout: osd deep scrub update digest min age = 30 2023-09-06T18:37:17.209 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.209 INFO:teuthology.orchestra.run.smithi099.stdout: osd map max advance = 10 2023-09-06T18:37:17.209 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.210 INFO:teuthology.orchestra.run.smithi099.stdout: journal zero on create = true 2023-09-06T18:37:17.210 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.210 INFO:teuthology.orchestra.run.smithi099.stdout: filestore ondisk finisher threads = 3 2023-09-06T18:37:17.210 INFO:teuthology.orchestra.run.smithi099.stdout: filestore apply finisher threads = 3 2023-09-06T18:37:17.210 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.210 INFO:teuthology.orchestra.run.smithi099.stdout: bdev debug aio = true 2023-09-06T18:37:17.211 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug misdirected ops = true 2023-09-06T18:37:17.211 INFO:teuthology.orchestra.run.smithi099.stdout: bdev async discard = True 2023-09-06T18:37:17.211 INFO:teuthology.orchestra.run.smithi099.stdout: bdev enable discard = True 2023-09-06T18:37:17.211 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore allocator = bitmap 2023-09-06T18:37:17.211 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore block size = 96636764160 2023-09-06T18:37:17.212 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore fsck on mount = True 2023-09-06T18:37:17.212 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluefs = 1/20 2023-09-06T18:37:17.212 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluestore = 1/20 2023-09-06T18:37:17.212 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2023-09-06T18:37:17.212 INFO:teuthology.orchestra.run.smithi099.stdout: debug osd = 20 2023-09-06T18:37:17.213 INFO:teuthology.orchestra.run.smithi099.stdout: debug rocksdb = 4/10 2023-09-06T18:37:17.213 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull_ratio = 0.85 2023-09-06T18:37:17.213 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.9 2023-09-06T18:37:17.213 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.8 2023-09-06T18:37:17.213 INFO:teuthology.orchestra.run.smithi099.stdout: osd failsafe full ratio = 0.95 2023-09-06T18:37:17.213 INFO:teuthology.orchestra.run.smithi099.stdout: osd objectstore = bluestore 2023-09-06T18:37:17.214 INFO:teuthology.orchestra.run.smithi099.stdout: osd shutdown pgref assert = True 2023-09-06T18:37:17.214 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.214 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr] 2023-09-06T18:37:17.214 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2023-09-06T18:37:17.214 INFO:teuthology.orchestra.run.smithi099.stdout: debug mgr = 20 2023-09-06T18:37:17.215 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2023-09-06T18:37:17.215 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2023-09-06T18:37:17.215 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2023-09-06T18:37:17.215 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2023-09-06T18:37:17.215 INFO:teuthology.orchestra.run.smithi099.stdout: mgr/telemetry/nag = false 2023-09-06T18:37:17.215 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.216 INFO:teuthology.orchestra.run.smithi099.stdout:[mon] 2023-09-06T18:37:17.216 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2023-09-06T18:37:17.216 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2023-09-06T18:37:17.216 INFO:teuthology.orchestra.run.smithi099.stdout: debug paxos = 20 2023-09-06T18:37:17.216 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2023-09-06T18:37:17.217 INFO:teuthology.orchestra.run.smithi099.stdout: mon data avail warn = 5 2023-09-06T18:37:17.217 INFO:teuthology.orchestra.run.smithi099.stdout: mon mgr mkfs grace = 240 2023-09-06T18:37:17.217 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2023-09-06T18:37:17.217 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd reporter subtree level = osd 2023-09-06T18:37:17.217 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd prime pg temp = true 2023-09-06T18:37:17.217 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2023-09-06T18:37:17.218 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.218 INFO:teuthology.orchestra.run.smithi099.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-06T18:37:17.218 INFO:teuthology.orchestra.run.smithi099.stdout: auth mon ticket ttl = 660 # 11m 2023-09-06T18:37:17.218 INFO:teuthology.orchestra.run.smithi099.stdout: auth service ticket ttl = 240 # 4m 2023-09-06T18:37:17.218 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.219 INFO:teuthology.orchestra.run.smithi099.stdout: # don't complain about insecure global_id in the test suite 2023-09-06T18:37:17.219 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-06T18:37:17.219 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-06T18:37:17.219 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.219 INFO:teuthology.orchestra.run.smithi099.stdout: # 1m isn't quite enough 2023-09-06T18:37:17.219 INFO:teuthology.orchestra.run.smithi099.stdout: mon_down_mkfs_grace = 2m 2023-09-06T18:37:17.220 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.220 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_filestore_osds = false 2023-09-06T18:37:17.220 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:17.220 INFO:teuthology.orchestra.run.smithi099.stdout:[client] 2023-09-06T18:37:17.220 INFO:teuthology.orchestra.run.smithi099.stdout: rgw cache enabled = true 2023-09-06T18:37:17.221 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable ops log = true 2023-09-06T18:37:17.221 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable usage log = true 2023-09-06T18:37:17.221 INFO:teuthology.orchestra.run.smithi099.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-06T18:37:17.221 INFO:teuthology.orchestra.run.smithi099.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-06T18:37:17.221 INFO:teuthology.orchestra.run.smithi099.stdout: debug rgw = 20 2023-09-06T18:37:17.222 INFO:teuthology.orchestra.run.smithi099.stdout: rgw crypt require ssl = False 2023-09-06T18:37:17.222 INFO:teuthology.orchestra.run.smithi099.stdout: rgw crypt s3 kms backend = testing 2023-09-06T18:37:17.222 INFO:teuthology.orchestra.run.smithi099.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2023-09-06T18:37:17.222 INFO:teuthology.orchestra.run.smithi099.stdout: rgw lc debug interval = 10 2023-09-06T18:37:17.222 INFO:teuthology.orchestra.run.smithi099.stdout: setgroup = ceph 2023-09-06T18:37:17.222 INFO:teuthology.orchestra.run.smithi099.stdout: setuser = ceph 2023-09-06T18:37:17.223 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.a] 2023-09-06T18:37:17.223 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.c] 2023-09-06T18:37:17.223 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.b] 2023-09-06T18:37:17.224 INFO:teuthology.orchestra.run.smithi181.stdout:[global] 2023-09-06T18:37:17.225 INFO:teuthology.orchestra.run.smithi181.stdout: chdir = "" 2023-09-06T18:37:17.225 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-06T18:37:17.225 INFO:teuthology.orchestra.run.smithi181.stdout: auth supported = cephx 2023-09-06T18:37:17.225 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.226 INFO:teuthology.orchestra.run.smithi181.stdout: filestore xattr use omap = true 2023-09-06T18:37:17.226 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.226 INFO:teuthology.orchestra.run.smithi181.stdout: mon clock drift allowed = 1.000 2023-09-06T18:37:17.226 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.226 INFO:teuthology.orchestra.run.smithi181.stdout: osd crush chooseleaf type = 0 2023-09-06T18:37:17.226 INFO:teuthology.orchestra.run.smithi181.stdout: auth debug = true 2023-09-06T18:37:17.227 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.227 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on old message = true 2023-09-06T18:37:17.227 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on bug = true 2023-09-06T18:37:17.227 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.227 INFO:teuthology.orchestra.run.smithi181.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-06T18:37:17.228 INFO:teuthology.orchestra.run.smithi181.stdout: mon pg warn max object skew = 0 2023-09-06T18:37:17.228 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.229 INFO:teuthology.orchestra.run.smithi181.stdout: # disable pg_autoscaler by default for new pools 2023-09-06T18:37:17.230 INFO:teuthology.orchestra.run.smithi181.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-06T18:37:17.230 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.230 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default size = 2 2023-09-06T18:37:17.230 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.230 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow primary affinity = true 2023-09-06T18:37:17.231 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow pg remap = true 2023-09-06T18:37:17.231 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on legacy crush tunables = false 2023-09-06T18:37:17.231 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on crush straw calc version zero = false 2023-09-06T18:37:17.231 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on no sortbitwise = false 2023-09-06T18:37:17.231 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on osd down out interval zero = false 2023-09-06T18:37:17.232 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on too few osds = false 2023-09-06T18:37:17.232 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-06T18:37:17.232 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-06T18:37:17.232 INFO:teuthology.orchestra.run.smithi181.stdout: mon_allow_pool_size_one = true 2023-09-06T18:37:17.232 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.232 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-06T18:37:17.233 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.233 INFO:teuthology.orchestra.run.smithi181.stdout: osd default data pool replay window = 5 2023-09-06T18:37:17.233 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.233 INFO:teuthology.orchestra.run.smithi181.stdout: mon allow pool delete = true 2023-09-06T18:37:17.233 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.234 INFO:teuthology.orchestra.run.smithi181.stdout: mon cluster log file level = debug 2023-09-06T18:37:17.234 INFO:teuthology.orchestra.run.smithi181.stdout: debug asserts on shutdown = true 2023-09-06T18:37:17.234 INFO:teuthology.orchestra.run.smithi181.stdout: mon health detail to clog = false 2023-09-06T18:37:17.234 INFO:teuthology.orchestra.run.smithi181.stdout: mon host = "172.21.15.99,[v2:172.21.15.99:3301,v1:172.21.15.99:6790],172.21.15.181" 2023-09-06T18:37:17.234 INFO:teuthology.orchestra.run.smithi181.stdout: osd_max_pg_log_entries = 10 2023-09-06T18:37:17.235 INFO:teuthology.orchestra.run.smithi181.stdout: osd_min_pg_log_entries = 10 2023-09-06T18:37:17.235 INFO:teuthology.orchestra.run.smithi181.stdout: fsid = b0ee29a8-5d11-45f6-ae33-fae8cc67c8f5 2023-09-06T18:37:17.235 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.235 INFO:teuthology.orchestra.run.smithi181.stdout:[osd] 2023-09-06T18:37:17.235 INFO:teuthology.orchestra.run.smithi181.stdout: osd journal size = 100 2023-09-06T18:37:17.236 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.236 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub load threshold = 5.0 2023-09-06T18:37:17.236 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub max interval = 600 2023-09-06T18:37:17.236 INFO:teuthology.orchestra.run.smithi181.stdout: osd mclock profile = high_recovery_ops 2023-09-06T18:37:17.236 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.236 INFO:teuthology.orchestra.run.smithi181.stdout: osd recover clone overlap = true 2023-09-06T18:37:17.237 INFO:teuthology.orchestra.run.smithi181.stdout: osd recovery max chunk = 1048576 2023-09-06T18:37:17.237 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.237 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug shutdown = true 2023-09-06T18:37:17.237 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug op order = true 2023-09-06T18:37:17.237 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug verify stray on activate = true 2023-09-06T18:37:17.238 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.238 INFO:teuthology.orchestra.run.smithi181.stdout: osd open classes on start = true 2023-09-06T18:37:17.238 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug pg log writeout = true 2023-09-06T18:37:17.238 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.238 INFO:teuthology.orchestra.run.smithi181.stdout: osd deep scrub update digest min age = 30 2023-09-06T18:37:17.239 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.239 INFO:teuthology.orchestra.run.smithi181.stdout: osd map max advance = 10 2023-09-06T18:37:17.239 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.239 INFO:teuthology.orchestra.run.smithi181.stdout: journal zero on create = true 2023-09-06T18:37:17.239 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.239 INFO:teuthology.orchestra.run.smithi181.stdout: filestore ondisk finisher threads = 3 2023-09-06T18:37:17.240 INFO:teuthology.orchestra.run.smithi181.stdout: filestore apply finisher threads = 3 2023-09-06T18:37:17.240 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.240 INFO:teuthology.orchestra.run.smithi181.stdout: bdev debug aio = true 2023-09-06T18:37:17.240 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug misdirected ops = true 2023-09-06T18:37:17.240 INFO:teuthology.orchestra.run.smithi181.stdout: bdev async discard = True 2023-09-06T18:37:17.241 INFO:teuthology.orchestra.run.smithi181.stdout: bdev enable discard = True 2023-09-06T18:37:17.241 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore allocator = bitmap 2023-09-06T18:37:17.241 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore block size = 96636764160 2023-09-06T18:37:17.241 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore fsck on mount = True 2023-09-06T18:37:17.241 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluefs = 1/20 2023-09-06T18:37:17.241 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluestore = 1/20 2023-09-06T18:37:17.242 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-09-06T18:37:17.242 INFO:teuthology.orchestra.run.smithi181.stdout: debug osd = 20 2023-09-06T18:37:17.242 INFO:teuthology.orchestra.run.smithi181.stdout: debug rocksdb = 4/10 2023-09-06T18:37:17.242 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd backfillfull_ratio = 0.85 2023-09-06T18:37:17.242 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd full ratio = 0.9 2023-09-06T18:37:17.243 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd nearfull ratio = 0.8 2023-09-06T18:37:17.243 INFO:teuthology.orchestra.run.smithi181.stdout: osd failsafe full ratio = 0.95 2023-09-06T18:37:17.243 INFO:teuthology.orchestra.run.smithi181.stdout: osd objectstore = bluestore 2023-09-06T18:37:17.243 INFO:teuthology.orchestra.run.smithi181.stdout: osd shutdown pgref assert = True 2023-09-06T18:37:17.243 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.244 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr] 2023-09-06T18:37:17.244 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-09-06T18:37:17.244 INFO:teuthology.orchestra.run.smithi181.stdout: debug mgr = 20 2023-09-06T18:37:17.244 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2023-09-06T18:37:17.244 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2023-09-06T18:37:17.244 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2023-09-06T18:37:17.245 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2023-09-06T18:37:17.245 INFO:teuthology.orchestra.run.smithi181.stdout: mgr/telemetry/nag = false 2023-09-06T18:37:17.245 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.245 INFO:teuthology.orchestra.run.smithi181.stdout:[mon] 2023-09-06T18:37:17.245 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-09-06T18:37:17.246 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2023-09-06T18:37:17.246 INFO:teuthology.orchestra.run.smithi181.stdout: debug paxos = 20 2023-09-06T18:37:17.246 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2023-09-06T18:37:17.246 INFO:teuthology.orchestra.run.smithi181.stdout: mon data avail warn = 5 2023-09-06T18:37:17.246 INFO:teuthology.orchestra.run.smithi181.stdout: mon mgr mkfs grace = 240 2023-09-06T18:37:17.247 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2023-09-06T18:37:17.247 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd reporter subtree level = osd 2023-09-06T18:37:17.247 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd prime pg temp = true 2023-09-06T18:37:17.247 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2023-09-06T18:37:17.247 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.248 INFO:teuthology.orchestra.run.smithi181.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-06T18:37:17.248 INFO:teuthology.orchestra.run.smithi181.stdout: auth mon ticket ttl = 660 # 11m 2023-09-06T18:37:17.248 INFO:teuthology.orchestra.run.smithi181.stdout: auth service ticket ttl = 240 # 4m 2023-09-06T18:37:17.248 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.248 INFO:teuthology.orchestra.run.smithi181.stdout: # don't complain about insecure global_id in the test suite 2023-09-06T18:37:17.249 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-06T18:37:17.249 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-06T18:37:17.249 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.249 INFO:teuthology.orchestra.run.smithi181.stdout: # 1m isn't quite enough 2023-09-06T18:37:17.249 INFO:teuthology.orchestra.run.smithi181.stdout: mon_down_mkfs_grace = 2m 2023-09-06T18:37:17.249 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.250 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_filestore_osds = false 2023-09-06T18:37:17.250 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-09-06T18:37:17.250 INFO:teuthology.orchestra.run.smithi181.stdout:[client] 2023-09-06T18:37:17.250 INFO:teuthology.orchestra.run.smithi181.stdout: rgw cache enabled = true 2023-09-06T18:37:17.250 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable ops log = true 2023-09-06T18:37:17.251 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable usage log = true 2023-09-06T18:37:17.251 INFO:teuthology.orchestra.run.smithi181.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-06T18:37:17.251 INFO:teuthology.orchestra.run.smithi181.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-06T18:37:17.251 INFO:teuthology.orchestra.run.smithi181.stdout: debug rgw = 20 2023-09-06T18:37:17.251 INFO:teuthology.orchestra.run.smithi181.stdout: rgw crypt require ssl = False 2023-09-06T18:37:17.252 INFO:teuthology.orchestra.run.smithi181.stdout: rgw crypt s3 kms backend = testing 2023-09-06T18:37:17.252 INFO:teuthology.orchestra.run.smithi181.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2023-09-06T18:37:17.252 INFO:teuthology.orchestra.run.smithi181.stdout: rgw lc debug interval = 10 2023-09-06T18:37:17.252 INFO:teuthology.orchestra.run.smithi181.stdout: setgroup = ceph 2023-09-06T18:37:17.252 INFO:teuthology.orchestra.run.smithi181.stdout: setuser = ceph 2023-09-06T18:37:17.252 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.a] 2023-09-06T18:37:17.253 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.c] 2023-09-06T18:37:17.253 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.b] 2023-09-06T18:37:17.253 INFO:tasks.ceph:Creating admin key on mon.a... 2023-09-06T18:37:17.254 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-09-06T18:37:17.321 INFO:tasks.ceph:Copying monmap to all nodes... 2023-09-06T18:37:17.322 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:17.322 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-09-06T18:37:17.340 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:17.340 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-09-06T18:37:17.398 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi099.front.sepia.ceph.com 2023-09-06T18:37:17.398 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:17.399 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-06T18:37:17.399 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-06T18:37:17.475 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:17.475 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-06T18:37:17.535 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi181.front.sepia.ceph.com 2023-09-06T18:37:17.535 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:17.536 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-06T18:37:17.536 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-06T18:37:17.573 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:17.574 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-06T18:37:17.632 INFO:tasks.ceph:Setting up mon nodes... 2023-09-06T18:37:17.633 INFO:tasks.ceph:Setting up mgr nodes... 2023-09-06T18:37:17.633 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-09-06T18:37:17.686 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-09-06T18:37:17.689 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-09-06T18:37:17.770 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-09-06T18:37:17.773 INFO:tasks.ceph:Setting up mds nodes... 2023-09-06T18:37:17.774 INFO:tasks.ceph_client:Setting up client nodes... 2023-09-06T18:37:17.774 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-06T18:37:17.819 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-09-06T18:37:17.831 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-09-06T18:37:17.875 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-09-06T18:37:17.888 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-09-06T18:37:17.888 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-09-06T18:37:17.889 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-09-06T18:37:17.916 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-06T18:37:17.916 INFO:tasks.ceph:role: osd.0 2023-09-06T18:37:17.917 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com 2023-09-06T18:37:17.917 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-06T18:37:17.990 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-06T18:37:17.991 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-06T18:37:17.991 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-06T18:37:17.991 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-06T18:37:17.991 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-06T18:37:17.992 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2023-09-06T18:37:17.992 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-06T18:37:17.992 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-06T18:37:17.992 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-06T18:37:17.992 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-06T18:37:18.420 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2023-09-06T18:37:18.421 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2023-09-06T18:37:18.422 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-09-06T18:37:18.602 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-09-06T18:37:18.639 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-09-06T18:37:18.708 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-06T18:37:18.708 INFO:tasks.ceph:role: osd.1 2023-09-06T18:37:18.708 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com 2023-09-06T18:37:18.709 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-06T18:37:18.783 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-06T18:37:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-06T18:37:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-06T18:37:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-06T18:37:18.784 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-06T18:37:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2023-09-06T18:37:18.784 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-06T18:37:18.785 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-06T18:37:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-06T18:37:18.785 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-06T18:37:19.216 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2023-09-06T18:37:19.218 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2023-09-06T18:37:19.218 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-09-06T18:37:19.321 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-09-06T18:37:19.353 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-09-06T18:37:19.420 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-06T18:37:19.421 INFO:tasks.ceph:role: osd.2 2023-09-06T18:37:19.421 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com 2023-09-06T18:37:19.421 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-06T18:37:19.495 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-06T18:37:19.495 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-06T18:37:19.495 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-06T18:37:19.496 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-06T18:37:19.496 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-06T18:37:19.496 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2023-09-06T18:37:19.496 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-06T18:37:19.497 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-06T18:37:19.497 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-06T18:37:19.497 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-06T18:37:19.939 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2023-09-06T18:37:19.940 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2023-09-06T18:37:19.941 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-09-06T18:37:20.073 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-09-06T18:37:20.109 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-09-06T18:37:20.177 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-06T18:37:20.178 INFO:tasks.ceph:role: osd.3 2023-09-06T18:37:20.178 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com 2023-09-06T18:37:20.178 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-06T18:37:20.252 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-06T18:37:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-06T18:37:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-06T18:37:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-06T18:37:20.253 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-06T18:37:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2023-09-06T18:37:20.254 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-06T18:37:20.254 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-06T18:37:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-06T18:37:20.255 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-06T18:37:20.705 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2023-09-06T18:37:20.707 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2023-09-06T18:37:20.707 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-09-06T18:37:20.893 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-09-06T18:37:20.925 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-06T18:37:21.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:21.021+0000 7fd4cf6a9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-09-06T18:37:21.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:21.021+0000 7fd4cf6a9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-09-06T18:37:21.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:21.021+0000 7fd4cf6a9640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-06T18:37:21.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:21.021+0000 7fd4cf6a9640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-06T18:37:21.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:21.021+0000 7fd4cf6a9640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-06T18:37:21.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:21.021+0000 7fd4cf6a9640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-09-06T18:37:23.876 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-09-06T18:37:23.909 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-06T18:37:24.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:24.000+0000 7faca5520640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-09-06T18:37:24.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:24.000+0000 7faca5520640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-09-06T18:37:24.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:24.000+0000 7faca5520640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-06T18:37:24.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:24.000+0000 7faca5520640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-06T18:37:24.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:24.000+0000 7faca5520640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-06T18:37:24.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:24.001+0000 7faca5520640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-09-06T18:37:26.864 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-09-06T18:37:26.897 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-06T18:37:26.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:26.985+0000 7f9f1a183640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-09-06T18:37:26.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:26.986+0000 7f9f1a183640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-09-06T18:37:26.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:26.986+0000 7f9f1a183640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-06T18:37:26.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:26.986+0000 7f9f1a183640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-06T18:37:26.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:26.986+0000 7f9f1a183640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-06T18:37:26.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:26.986+0000 7f9f1a183640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-09-06T18:37:29.838 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-09-06T18:37:29.868 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-06T18:37:29.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:29.955+0000 7f53080d1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-09-06T18:37:29.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:29.956+0000 7f53080d1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-09-06T18:37:29.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:29.956+0000 7f53080d1640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-06T18:37:29.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:29.956+0000 7f53080d1640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-06T18:37:29.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:29.956+0000 7f53080d1640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-06T18:37:29.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:37:29.956+0000 7f53080d1640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-09-06T18:37:32.819 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-09-06T18:37:32.852 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-09-06T18:37:32.887 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-06T18:37:32.887 INFO:tasks.ceph:role: osd.4 2023-09-06T18:37:32.887 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com 2023-09-06T18:37:32.888 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-06T18:37:32.965 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-06T18:37:32.966 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-06T18:37:32.966 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-06T18:37:32.966 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-06T18:37:32.966 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-06T18:37:32.967 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-09-06T18:37:32.967 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-06T18:37:32.967 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-06T18:37:32.967 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-06T18:37:32.967 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-06T18:37:33.394 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-09-06T18:37:33.396 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-09-06T18:37:33.397 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-09-06T18:37:33.593 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-09-06T18:37:33.632 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-09-06T18:37:33.705 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-06T18:37:33.705 INFO:tasks.ceph:role: osd.5 2023-09-06T18:37:33.706 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com 2023-09-06T18:37:33.706 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-06T18:37:33.782 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-06T18:37:33.782 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-06T18:37:33.782 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-06T18:37:33.782 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-06T18:37:33.783 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-06T18:37:33.783 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-09-06T18:37:33.783 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-06T18:37:33.783 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-06T18:37:33.783 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-06T18:37:33.784 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-06T18:37:34.198 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-09-06T18:37:34.200 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-09-06T18:37:34.200 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-09-06T18:37:34.318 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-09-06T18:37:34.353 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-09-06T18:37:34.422 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-06T18:37:34.422 INFO:tasks.ceph:role: osd.6 2023-09-06T18:37:34.423 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com 2023-09-06T18:37:34.423 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-06T18:37:34.499 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-06T18:37:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-06T18:37:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-06T18:37:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-06T18:37:34.500 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-06T18:37:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-09-06T18:37:34.500 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-06T18:37:34.500 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-06T18:37:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-06T18:37:34.501 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-06T18:37:34.954 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-09-06T18:37:34.956 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-09-06T18:37:34.957 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-09-06T18:37:35.119 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-09-06T18:37:35.156 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-09-06T18:37:35.226 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-06T18:37:35.226 INFO:tasks.ceph:role: osd.7 2023-09-06T18:37:35.227 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com 2023-09-06T18:37:35.227 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-06T18:37:35.302 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-06T18:37:35.302 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-06T18:37:35.303 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-06T18:37:35.303 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-06T18:37:35.303 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-06T18:37:35.303 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-09-06T18:37:35.303 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-06T18:37:35.304 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-06T18:37:35.304 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-06T18:37:35.304 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-06T18:37:35.737 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-09-06T18:37:35.738 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-09-06T18:37:35.739 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-09-06T18:37:35.899 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-09-06T18:37:35.934 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-06T18:37:36.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:36.035+0000 7ff0bde7c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-09-06T18:37:36.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:36.036+0000 7ff0bde7c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-09-06T18:37:36.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:36.036+0000 7ff0bde7c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-06T18:37:36.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:36.036+0000 7ff0bde7c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-06T18:37:36.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:36.036+0000 7ff0bde7c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-06T18:37:36.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:36.036+0000 7ff0bde7c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-09-06T18:37:38.882 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-09-06T18:37:38.914 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-06T18:37:39.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:39.005+0000 7fc90be83640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-09-06T18:37:39.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:39.005+0000 7fc90be83640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-09-06T18:37:39.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:39.005+0000 7fc90be83640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-06T18:37:39.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:39.005+0000 7fc90be83640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-06T18:37:39.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:39.005+0000 7fc90be83640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-06T18:37:39.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:39.006+0000 7fc90be83640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-09-06T18:37:41.859 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-09-06T18:37:41.891 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-06T18:37:41.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:41.984+0000 7f2e4ef40640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-09-06T18:37:41.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:41.984+0000 7f2e4ef40640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-09-06T18:37:41.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:41.984+0000 7f2e4ef40640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-06T18:37:41.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:41.984+0000 7f2e4ef40640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-06T18:37:41.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:41.984+0000 7f2e4ef40640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-06T18:37:41.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:41.984+0000 7f2e4ef40640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-09-06T18:37:44.835 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-09-06T18:37:44.867 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-06T18:37:44.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:44.961+0000 7f561a428640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-09-06T18:37:44.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:44.961+0000 7f561a428640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-09-06T18:37:44.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:44.961+0000 7f561a428640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-06T18:37:44.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:44.961+0000 7f561a428640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-06T18:37:44.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:44.961+0000 7f561a428640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-06T18:37:44.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-09-06T18:37:44.962+0000 7f561a428640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-09-06T18:37:47.811 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-09-06T18:37:47.843 INFO:tasks.ceph:Reading keys from all nodes... 2023-09-06T18:37:47.844 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:47.844 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-09-06T18:37:47.880 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:47.880 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-09-06T18:37:47.951 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:47.951 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-09-06T18:37:48.022 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:48.022 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-09-06T18:37:48.091 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:48.091 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-09-06T18:37:48.160 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:48.160 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-09-06T18:37:48.193 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:48.194 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-09-06T18:37:48.265 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:48.266 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-09-06T18:37:48.337 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:48.337 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-09-06T18:37:48.406 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:48.407 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-09-06T18:37:48.476 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:48.476 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-09-06T18:37:48.500 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:48.500 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-09-06T18:37:48.535 INFO:tasks.ceph:Adding keys to all mons... 2023-09-06T18:37:48.536 DEBUG:teuthology.orchestra.run.smithi099:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-06T18:37:48.543 DEBUG:teuthology.orchestra.run.smithi181:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-06T18:37:48.579 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.y] 2023-09-06T18:37:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDdxvhkuLDwKBAAYzTqOFGbUlDnYgcsMt7tmA== 2023-09-06T18:37:48.580 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.0] 2023-09-06T18:37:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDhxvhkOx1ZARAAINs6pnCBtO3kbPehd7kbhw== 2023-09-06T18:37:48.580 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.1] 2023-09-06T18:37:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDkxvhkV6QeABAAQrStpLtdrFdiravORoIUPw== 2023-09-06T18:37:48.580 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.2] 2023-09-06T18:37:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDmxvhk+ZnYOhAAT57y4WgRjLKsIQYjTAasYQ== 2023-09-06T18:37:48.581 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.3] 2023-09-06T18:37:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDpxvhkAiEQORAAT+PBhTs6DBGB6lvkeSSeuQ== 2023-09-06T18:37:48.581 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.x] 2023-09-06T18:37:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDdxvhkL43xLRAAYqNPTuh2NFpNkFhuN7kRBw== 2023-09-06T18:37:48.582 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.4] 2023-09-06T18:37:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDwxvhkZqgwAhAAtqiRYrBom2jZ3APa3GItMg== 2023-09-06T18:37:48.582 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.5] 2023-09-06T18:37:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDzxvhkDPFxABAA6flrwkZ7HYNO3eur8ctGyg== 2023-09-06T18:37:48.582 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.6] 2023-09-06T18:37:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQD1xvhkEHbDOhAAsBh1LgrkypiAFKWx+lVS0Q== 2023-09-06T18:37:48.583 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.7] 2023-09-06T18:37:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQD4xvhkQmldORAA42LpQKxVGzWi0F4J79qCSg== 2023-09-06T18:37:48.583 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2023-09-06T18:37:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDdxvhkiPnQMBAA5ELEuxPn2FDN86TJxrxJKw== 2023-09-06T18:37:48.584 INFO:teuthology.orchestra.run.smithi099.stdout:[client.1] 2023-09-06T18:37:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDdxvhknAcsNBAA5ZnfXLrofFbr3rx8FVeFzg== 2023-09-06T18:37:48.602 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.y] 2023-09-06T18:37:48.602 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDdxvhkuLDwKBAAYzTqOFGbUlDnYgcsMt7tmA== 2023-09-06T18:37:48.603 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.0] 2023-09-06T18:37:48.603 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDhxvhkOx1ZARAAINs6pnCBtO3kbPehd7kbhw== 2023-09-06T18:37:48.603 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.1] 2023-09-06T18:37:48.603 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDkxvhkV6QeABAAQrStpLtdrFdiravORoIUPw== 2023-09-06T18:37:48.603 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.2] 2023-09-06T18:37:48.604 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDmxvhk+ZnYOhAAT57y4WgRjLKsIQYjTAasYQ== 2023-09-06T18:37:48.604 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.3] 2023-09-06T18:37:48.604 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDpxvhkAiEQORAAT+PBhTs6DBGB6lvkeSSeuQ== 2023-09-06T18:37:48.604 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.x] 2023-09-06T18:37:48.605 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDdxvhkL43xLRAAYqNPTuh2NFpNkFhuN7kRBw== 2023-09-06T18:37:48.605 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.4] 2023-09-06T18:37:48.605 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDwxvhkZqgwAhAAtqiRYrBom2jZ3APa3GItMg== 2023-09-06T18:37:48.605 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.5] 2023-09-06T18:37:48.605 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDzxvhkDPFxABAA6flrwkZ7HYNO3eur8ctGyg== 2023-09-06T18:37:48.605 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.6] 2023-09-06T18:37:48.606 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQD1xvhkEHbDOhAAsBh1LgrkypiAFKWx+lVS0Q== 2023-09-06T18:37:48.606 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.7] 2023-09-06T18:37:48.606 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQD4xvhkQmldORAA42LpQKxVGzWi0F4J79qCSg== 2023-09-06T18:37:48.606 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-09-06T18:37:48.606 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDdxvhkiPnQMBAA5ELEuxPn2FDN86TJxrxJKw== 2023-09-06T18:37:48.607 INFO:teuthology.orchestra.run.smithi181.stdout:[client.1] 2023-09-06T18:37:48.607 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDdxvhknAcsNBAA5ZnfXLrofFbr3rx8FVeFzg== 2023-09-06T18:37:48.608 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-06T18:37:48.628 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-06T18:37:48.702 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:48.705 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:48.759 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:48.762 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:48.815 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:48.817 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:48.871 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:48.875 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:48.930 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-06T18:37:48.932 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-06T18:37:48.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:48.989 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:49.043 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:49.045 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:49.100 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:49.102 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:49.156 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:49.158 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-06T18:37:49.212 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-06T18:37:49.214 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-06T18:37:49.272 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-06T18:37:49.275 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-06T18:37:49.333 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-09-06T18:37:49.334 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-09-06T18:37:49.362 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-06T18:37:49.460 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-09-06T18:37:49.487 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-09-06T18:37:49.554 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-06T18:37:49.653 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-09-06T18:37:49.680 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-09-06T18:37:49.709 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-06T18:37:49.810 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-09-06T18:37:49.838 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-06T18:37:49.840 DEBUG:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-06T18:37:49.895 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-09-06T18:37:49.896 INFO:tasks.ceph.mon.a:Restarting daemon 2023-09-06T18:37:49.896 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-09-06T18:37:49.899 INFO:tasks.ceph.mon.a:Started 2023-09-06T18:37:49.899 INFO:tasks.ceph.mon.c:Restarting daemon 2023-09-06T18:37:49.899 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-09-06T18:37:49.901 INFO:tasks.ceph.mon.c:Started 2023-09-06T18:37:49.901 INFO:tasks.ceph.mon.b:Restarting daemon 2023-09-06T18:37:49.902 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-09-06T18:37:49.938 INFO:tasks.ceph.mon.b:Started 2023-09-06T18:37:49.938 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-09-06T18:37:49.939 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-09-06T18:37:49.939 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-09-06T18:37:49.941 INFO:tasks.ceph.mgr.y:Started 2023-09-06T18:37:49.941 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-06T18:37:49.942 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-06T18:37:49.944 INFO:tasks.ceph.mgr.x:Started 2023-09-06T18:37:49.944 INFO:tasks.ceph:Setting crush tunables to default 2023-09-06T18:37:49.944 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd crush tunables default 2023-09-06T18:37:50.351 INFO:teuthology.orchestra.run.smithi099.stderr:adjusted tunables profile to default 2023-09-06T18:37:50.362 INFO:tasks.ceph:check_enable_crimson: False 2023-09-06T18:37:50.362 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-09-06T18:37:50.363 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:50.363 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-09-06T18:37:50.380 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:50.380+0000 7ff6f3079200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-06T18:37:50.390 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:50.391 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-09-06T18:37:50.457 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:50.457 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-09-06T18:37:50.524 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:37:50.525 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-09-06T18:37:50.558 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:50.557+0000 7ff6f3079200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-06T18:37:50.591 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:50.591 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-09-06T18:37:50.619 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:50.619 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-09-06T18:37:50.686 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:50.686 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-09-06T18:37:50.750 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:50.749+0000 7fea078c0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-06T18:37:50.753 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-09-06T18:37:50.754 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-09-06T18:37:50.823 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 60cafc8d-10aa-4647-8938-1cd7313d640c 0 2023-09-06T18:37:50.923 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:50.923+0000 7fea078c0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-06T18:37:50.962 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:50.961+0000 7ff6f3079200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-06T18:37:51.086 INFO:teuthology.orchestra.run.smithi181.stdout:0 2023-09-06T18:37:51.099 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new d5358f90-6305-4897-870a-8d1a4f51ff85 1 2023-09-06T18:37:51.295 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:51.294+0000 7fea078c0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-06T18:37:51.330 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-09-06T18:37:51.343 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new edaaeed8-d9f2-4faf-aa0d-b4cf99422d69 2 2023-09-06T18:37:51.474 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:51.473+0000 7ff6f3079200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-06T18:37:51.563 INFO:teuthology.orchestra.run.smithi181.stdout:2 2023-09-06T18:37:51.575 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 81e09a9c-fb99-428e-8b78-c81e9362e7bd 3 2023-09-06T18:37:51.678 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:51.677+0000 7ff6f3079200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-06T18:37:51.785 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:51.784+0000 7fea078c0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-06T18:37:51.799 INFO:teuthology.orchestra.run.smithi181.stdout:3 2023-09-06T18:37:51.812 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 9ebe4914-2849-4aa4-ba9a-ffd53485ca46 4 2023-09-06T18:37:51.978 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:51.978+0000 7fea078c0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-06T18:37:52.068 INFO:teuthology.orchestra.run.smithi181.stdout:4 2023-09-06T18:37:52.080 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new b0985b70-1d44-46b3-8e13-1f4ae4cf89b3 5 2023-09-06T18:37:52.099 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:52.098+0000 7ff6f3079200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-06T18:37:52.301 INFO:teuthology.orchestra.run.smithi181.stdout:5 2023-09-06T18:37:52.313 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 7fe89e17-ee50-4832-827e-f770f2a97371 6 2023-09-06T18:37:52.363 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:52.362+0000 7fea078c0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-06T18:37:52.572 INFO:teuthology.orchestra.run.smithi181.stdout:6 2023-09-06T18:37:52.585 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new fb42cb6a-52fc-434a-b6cf-616d7fc470e1 7 2023-09-06T18:37:52.794 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:52.793+0000 7ff6f3079200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-06T18:37:52.808 INFO:teuthology.orchestra.run.smithi181.stdout:7 2023-09-06T18:37:52.821 INFO:tasks.ceph.osd.0:Restarting daemon 2023-09-06T18:37:52.821 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-09-06T18:37:52.824 INFO:tasks.ceph.osd.0:Started 2023-09-06T18:37:52.824 INFO:tasks.ceph.osd.1:Restarting daemon 2023-09-06T18:37:52.824 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-09-06T18:37:52.827 INFO:tasks.ceph.osd.1:Started 2023-09-06T18:37:52.827 INFO:tasks.ceph.osd.2:Restarting daemon 2023-09-06T18:37:52.827 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-09-06T18:37:52.829 INFO:tasks.ceph.osd.2:Started 2023-09-06T18:37:52.830 INFO:tasks.ceph.osd.3:Restarting daemon 2023-09-06T18:37:52.830 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-09-06T18:37:52.832 INFO:tasks.ceph.osd.3:Started 2023-09-06T18:37:52.833 INFO:tasks.ceph.osd.4:Restarting daemon 2023-09-06T18:37:52.833 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-09-06T18:37:52.835 INFO:tasks.ceph.osd.4:Started 2023-09-06T18:37:52.836 INFO:tasks.ceph.osd.5:Restarting daemon 2023-09-06T18:37:52.836 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-09-06T18:37:52.838 INFO:tasks.ceph.osd.5:Started 2023-09-06T18:37:52.838 INFO:tasks.ceph.osd.6:Restarting daemon 2023-09-06T18:37:52.838 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-09-06T18:37:52.841 INFO:tasks.ceph.osd.6:Started 2023-09-06T18:37:52.841 INFO:tasks.ceph.osd.7:Restarting daemon 2023-09-06T18:37:52.841 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-09-06T18:37:52.847 INFO:tasks.ceph.osd.7:Started 2023-09-06T18:37:52.847 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-06T18:37:52.881 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:52.880+0000 7ff6f3079200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-06T18:37:53.035 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:53.034+0000 7ff6f3079200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-06T18:37:53.073 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:53.072+0000 7fea078c0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-06T18:37:53.073 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:37:53.074 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":10,"fsid":"b0ee29a8-5d11-45f6-ae33-fae8cc67c8f5","created":"2023-09-06T18:37:50.238993+0000","modified":"2023-09-06T18:37:52.805129+0000","last_up_change":"0.000000","last_in_change":"2023-09-06T18:37:52.805129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"60cafc8d-10aa-4647-8938-1cd7313d640c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d5358f90-6305-4897-870a-8d1a4f51ff85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"edaaeed8-d9f2-4faf-aa0d-b4cf99422d69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"81e09a9c-fb99-428e-8b78-c81e9362e7bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9ebe4914-2849-4aa4-ba9a-ffd53485ca46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b0985b70-1d44-46b3-8e13-1f4ae4cf89b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7fe89e17-ee50-4832-827e-f770f2a97371","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fb42cb6a-52fc-434a-b6cf-616d7fc470e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-06T18:37:53.086 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-09-06T18:37:53.086 INFO:tasks.ceph:Waiting for OSDs to come up 2023-09-06T18:37:53.104 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:53.103+0000 7ff6f3079200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-06T18:37:53.148 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:53.147+0000 7fea078c0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-06T18:37:53.285 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:53.285+0000 7fea078c0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-06T18:37:53.300 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:53.299+0000 7ff6f3079200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-06T18:37:53.355 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:53.355+0000 7fea078c0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-06T18:37:53.378 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:53.377+0000 7ff6f3079200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-06T18:37:53.437 INFO:tasks.ceph.osd.0.smithi099.stderr:2023-09-06T18:37:53.436+0000 7fb98467b640 -1 Falling back to public interface 2023-09-06T18:37:53.442 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-09-06T18:37:53.441+0000 7eff255cf640 -1 Falling back to public interface 2023-09-06T18:37:53.454 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-09-06T18:37:53.453+0000 7f148e3d5640 -1 Falling back to public interface 2023-09-06T18:37:53.456 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:53.455+0000 7ff6f3079200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-06T18:37:53.463 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-09-06T18:37:53.462+0000 7fe8c5c45640 -1 Falling back to public interface 2023-09-06T18:37:53.467 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-09-06T18:37:53.466+0000 7ffb1234b640 -1 Falling back to public interface 2023-09-06T18:37:53.475 INFO:tasks.ceph.osd.1.smithi099.stderr:2023-09-06T18:37:53.474+0000 7f41aa425640 -1 Falling back to public interface 2023-09-06T18:37:53.480 INFO:tasks.ceph.osd.3.smithi099.stderr:2023-09-06T18:37:53.479+0000 7fbc78a19640 -1 Falling back to public interface 2023-09-06T18:37:53.480 INFO:tasks.ceph.osd.2.smithi099.stderr:2023-09-06T18:37:53.479+0000 7f47155f1640 -1 Falling back to public interface 2023-09-06T18:37:53.541 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:53.540+0000 7fea078c0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-06T18:37:53.549 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:53.548+0000 7ff6f3079200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-06T18:37:53.619 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:53.618+0000 7fea078c0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-06T18:37:53.695 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:53.695+0000 7fea078c0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-06T18:37:53.732 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:53.731+0000 7ff6f3079200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-06T18:37:53.788 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:53.788+0000 7fea078c0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-06T18:37:53.891 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-06T18:37:53.965 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:53.964+0000 7ff6f3079200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-06T18:37:53.972 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:53.971+0000 7fea078c0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-06T18:37:54.085 INFO:teuthology.misc.health.smithi099.stdout: 2023-09-06T18:37:54.086 INFO:teuthology.misc.health.smithi099.stdout:{"epoch":10,"fsid":"b0ee29a8-5d11-45f6-ae33-fae8cc67c8f5","created":"2023-09-06T18:37:50.238993+0000","modified":"2023-09-06T18:37:52.805129+0000","last_up_change":"0.000000","last_in_change":"2023-09-06T18:37:52.805129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"60cafc8d-10aa-4647-8938-1cd7313d640c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d5358f90-6305-4897-870a-8d1a4f51ff85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"edaaeed8-d9f2-4faf-aa0d-b4cf99422d69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"81e09a9c-fb99-428e-8b78-c81e9362e7bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9ebe4914-2849-4aa4-ba9a-ffd53485ca46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b0985b70-1d44-46b3-8e13-1f4ae4cf89b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7fe89e17-ee50-4832-827e-f770f2a97371","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fb42cb6a-52fc-434a-b6cf-616d7fc470e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-06T18:37:54.097 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-09-06T18:37:54.121 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:54.120+0000 7ff6f3079200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-06T18:37:54.193 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:54.192+0000 7ff6f3079200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-06T18:37:54.194 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:54.194+0000 7fea078c0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-06T18:37:54.350 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:54.350+0000 7fea078c0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-06T18:37:54.420 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:54.420+0000 7fea078c0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-06T18:37:54.712 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:54.711+0000 7ff6f3079200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-06T18:37:54.939 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:54.939+0000 7fea078c0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-06T18:37:55.034 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:55.032+0000 7ff6f3079200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-06T18:37:55.181 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:55.180+0000 7ff6f3079200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-06T18:37:55.256 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:55.255+0000 7fea078c0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-06T18:37:55.401 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:55.400+0000 7fea078c0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-06T18:37:55.578 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:55.577+0000 7ff6f3079200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-06T18:37:55.728 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:55.727+0000 7ff6f3079200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-06T18:37:55.799 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:55.799+0000 7fea078c0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-06T18:37:55.879 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:55.878+0000 7ff6f3079200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-06T18:37:55.956 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:55.955+0000 7fea078c0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-06T18:37:56.106 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:56.106+0000 7fea078c0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-06T18:37:56.219 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:56.218+0000 7ff6f3079200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-06T18:37:56.494 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-09-06T18:37:56.493+0000 7fea078c0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-06T18:37:56.931 INFO:tasks.ceph.osd.0.smithi099.stderr:2023-09-06T18:37:56.930+0000 7fb98467b640 -1 osd.0 0 log_to_monitors true 2023-09-06T18:37:56.939 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-09-06T18:37:56.938+0000 7eff255cf640 -1 osd.4 0 log_to_monitors true 2023-09-06T18:37:56.948 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-09-06T18:37:56.947+0000 7f148e3d5640 -1 osd.7 0 log_to_monitors true 2023-09-06T18:37:56.967 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-09-06T18:37:56.966+0000 7fe8c5c45640 -1 osd.5 0 log_to_monitors true 2023-09-06T18:37:56.989 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-09-06T18:37:56.988+0000 7ffb1234b640 -1 osd.6 0 log_to_monitors true 2023-09-06T18:37:56.999 INFO:tasks.ceph.osd.2.smithi099.stderr:2023-09-06T18:37:56.998+0000 7f47155f1640 -1 osd.2 0 log_to_monitors true 2023-09-06T18:37:57.005 INFO:tasks.ceph.osd.3.smithi099.stderr:2023-09-06T18:37:57.005+0000 7fbc78a19640 -1 osd.3 0 log_to_monitors true 2023-09-06T18:37:57.008 INFO:tasks.ceph.osd.1.smithi099.stderr:2023-09-06T18:37:57.004+0000 7f41aa425640 -1 osd.1 0 log_to_monitors true 2023-09-06T18:37:57.247 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:57.246+0000 7ff6b446b700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-09-06T18:37:57.248 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-09-06T18:37:57.247+0000 7ff6b446b700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-06T18:38:00.289 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-09-06T18:38:00.288+0000 7ffb05eeb700 -1 osd.6 0 waiting for initial osdmap 2023-09-06T18:38:00.302 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-09-06T18:38:00.301+0000 7ffafe2c6700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-06T18:38:00.306 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-09-06T18:38:00.306+0000 7eff1916f700 -1 osd.4 0 waiting for initial osdmap 2023-09-06T18:38:00.326 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-09-06T18:38:00.325+0000 7eff1154a700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-06T18:38:00.362 INFO:tasks.ceph.osd.1.smithi099.stderr:2023-09-06T18:38:00.361+0000 7f419dfc5700 -1 osd.1 0 waiting for initial osdmap 2023-09-06T18:38:00.377 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-09-06T18:38:00.376+0000 7fe8b97e5700 -1 osd.5 0 waiting for initial osdmap 2023-09-06T18:38:00.381 INFO:tasks.ceph.osd.1.smithi099.stderr:2023-09-06T18:38:00.380+0000 7f41963a0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-06T18:38:00.382 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-09-06T18:38:00.381+0000 7f1481f75700 -1 osd.7 0 waiting for initial osdmap 2023-09-06T18:38:00.384 INFO:tasks.ceph.osd.3.smithi099.stderr:2023-09-06T18:38:00.384+0000 7fbc6c5b9700 -1 osd.3 0 waiting for initial osdmap 2023-09-06T18:38:00.393 INFO:tasks.ceph.osd.3.smithi099.stderr:2023-09-06T18:38:00.393+0000 7fbc64994700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-06T18:38:00.396 INFO:tasks.ceph.osd.2.smithi099.stderr:2023-09-06T18:38:00.396+0000 7f4709191700 -1 osd.2 0 waiting for initial osdmap 2023-09-06T18:38:00.398 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-09-06T18:38:00.397+0000 7fe8b1bc0700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-06T18:38:00.402 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-09-06T18:38:00.401+0000 7f147a350700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-06T18:38:00.405 INFO:tasks.ceph.osd.0.smithi099.stderr:2023-09-06T18:38:00.405+0000 7fb97821b700 -1 osd.0 0 waiting for initial osdmap 2023-09-06T18:38:00.411 INFO:tasks.ceph.osd.2.smithi099.stderr:2023-09-06T18:38:00.411+0000 7f470156c700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-06T18:38:00.419 INFO:tasks.ceph.osd.0.smithi099.stderr:2023-09-06T18:38:00.419+0000 7fb9705f6700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-06T18:38:00.904 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-06T18:38:01.214 INFO:teuthology.misc.health.smithi099.stdout: 2023-09-06T18:38:01.215 INFO:teuthology.misc.health.smithi099.stdout:{"epoch":12,"fsid":"b0ee29a8-5d11-45f6-ae33-fae8cc67c8f5","created":"2023-09-06T18:37:50.238993+0000","modified":"2023-09-06T18:37:58.320680+0000","last_up_change":"0.000000","last_in_change":"2023-09-06T18:37:52.805129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"60cafc8d-10aa-4647-8938-1cd7313d640c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d5358f90-6305-4897-870a-8d1a4f51ff85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"edaaeed8-d9f2-4faf-aa0d-b4cf99422d69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"81e09a9c-fb99-428e-8b78-c81e9362e7bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9ebe4914-2849-4aa4-ba9a-ffd53485ca46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b0985b70-1d44-46b3-8e13-1f4ae4cf89b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7fe89e17-ee50-4832-827e-f770f2a97371","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fb42cb6a-52fc-434a-b6cf-616d7fc470e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-06T18:38:01.227 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-09-06T18:38:08.035 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-06T18:38:08.360 INFO:teuthology.misc.health.smithi099.stdout: 2023-09-06T18:38:08.360 INFO:teuthology.misc.health.smithi099.stdout:{"epoch":16,"fsid":"b0ee29a8-5d11-45f6-ae33-fae8cc67c8f5","created":"2023-09-06T18:37:50.238993+0000","modified":"2023-09-06T18:38:04.541690+0000","last_up_change":"2023-09-06T18:38:01.304703+0000","last_in_change":"2023-09-06T18:37:52.805129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-06T18:38:02.303794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"60cafc8d-10aa-4647-8938-1cd7313d640c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2410739650},{"type":"v1","addr":"172.21.15.99:6801","nonce":2410739650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2410739650},{"type":"v1","addr":"172.21.15.99:6803","nonce":2410739650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2410739650},{"type":"v1","addr":"172.21.15.99:6807","nonce":2410739650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2410739650},{"type":"v1","addr":"172.21.15.99:6805","nonce":2410739650}]},"public_addr":"172.21.15.99:6801/2410739650","cluster_addr":"172.21.15.99:6803/2410739650","heartbeat_back_addr":"172.21.15.99:6807/2410739650","heartbeat_front_addr":"172.21.15.99:6805/2410739650","state":["exists","up"]},{"osd":1,"uuid":"d5358f90-6305-4897-870a-8d1a4f51ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1561643328},{"type":"v1","addr":"172.21.15.99:6809","nonce":1561643328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1561643328},{"type":"v1","addr":"172.21.15.99:6811","nonce":1561643328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1561643328},{"type":"v1","addr":"172.21.15.99:6815","nonce":1561643328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1561643328},{"type":"v1","addr":"172.21.15.99:6813","nonce":1561643328}]},"public_addr":"172.21.15.99:6809/1561643328","cluster_addr":"172.21.15.99:6811/1561643328","heartbeat_back_addr":"172.21.15.99:6815/1561643328","heartbeat_front_addr":"172.21.15.99:6813/1561643328","state":["exists","up"]},{"osd":2,"uuid":"edaaeed8-d9f2-4faf-aa0d-b4cf99422d69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1188692910},{"type":"v1","addr":"172.21.15.99:6819","nonce":1188692910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":1188692910},{"type":"v1","addr":"172.21.15.99:6823","nonce":1188692910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6829","nonce":1188692910},{"type":"v1","addr":"172.21.15.99:6831","nonce":1188692910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1188692910},{"type":"v1","addr":"172.21.15.99:6827","nonce":1188692910}]},"public_addr":"172.21.15.99:6819/1188692910","cluster_addr":"172.21.15.99:6823/1188692910","heartbeat_back_addr":"172.21.15.99:6831/1188692910","heartbeat_front_addr":"172.21.15.99:6827/1188692910","state":["exists","up"]},{"osd":3,"uuid":"81e09a9c-fb99-428e-8b78-c81e9362e7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2909409728},{"type":"v1","addr":"172.21.15.99:6818","nonce":2909409728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2909409728},{"type":"v1","addr":"172.21.15.99:6822","nonce":2909409728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2909409728},{"type":"v1","addr":"172.21.15.99:6830","nonce":2909409728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2909409728},{"type":"v1","addr":"172.21.15.99:6826","nonce":2909409728}]},"public_addr":"172.21.15.99:6818/2909409728","cluster_addr":"172.21.15.99:6822/2909409728","heartbeat_back_addr":"172.21.15.99:6830/2909409728","heartbeat_front_addr":"172.21.15.99:6826/2909409728","state":["exists","up"]},{"osd":4,"uuid":"9ebe4914-2849-4aa4-ba9a-ffd53485ca46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":520119356},{"type":"v1","addr":"172.21.15.181:6801","nonce":520119356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":520119356},{"type":"v1","addr":"172.21.15.181:6803","nonce":520119356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":520119356},{"type":"v1","addr":"172.21.15.181:6807","nonce":520119356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":520119356},{"type":"v1","addr":"172.21.15.181:6805","nonce":520119356}]},"public_addr":"172.21.15.181:6801/520119356","cluster_addr":"172.21.15.181:6803/520119356","heartbeat_back_addr":"172.21.15.181:6807/520119356","heartbeat_front_addr":"172.21.15.181:6805/520119356","state":["exists","up"]},{"osd":5,"uuid":"b0985b70-1d44-46b3-8e13-1f4ae4cf89b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3909493596},{"type":"v1","addr":"172.21.15.181:6817","nonce":3909493596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3909493596},{"type":"v1","addr":"172.21.15.181:6819","nonce":3909493596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3909493596},{"type":"v1","addr":"172.21.15.181:6823","nonce":3909493596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3909493596},{"type":"v1","addr":"172.21.15.181:6821","nonce":3909493596}]},"public_addr":"172.21.15.181:6817/3909493596","cluster_addr":"172.21.15.181:6819/3909493596","heartbeat_back_addr":"172.21.15.181:6823/3909493596","heartbeat_front_addr":"172.21.15.181:6821/3909493596","state":["exists","up"]},{"osd":6,"uuid":"7fe89e17-ee50-4832-827e-f770f2a97371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2835950561},{"type":"v1","addr":"172.21.15.181:6825","nonce":2835950561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2835950561},{"type":"v1","addr":"172.21.15.181:6827","nonce":2835950561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2835950561},{"type":"v1","addr":"172.21.15.181:6831","nonce":2835950561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2835950561},{"type":"v1","addr":"172.21.15.181:6829","nonce":2835950561}]},"public_addr":"172.21.15.181:6825/2835950561","cluster_addr":"172.21.15.181:6827/2835950561","heartbeat_back_addr":"172.21.15.181:6831/2835950561","heartbeat_front_addr":"172.21.15.181:6829/2835950561","state":["exists","up"]},{"osd":7,"uuid":"fb42cb6a-52fc-434a-b6cf-616d7fc470e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2848929566},{"type":"v1","addr":"172.21.15.181:6809","nonce":2848929566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2848929566},{"type":"v1","addr":"172.21.15.181:6811","nonce":2848929566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2848929566},{"type":"v1","addr":"172.21.15.181:6815","nonce":2848929566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2848929566},{"type":"v1","addr":"172.21.15.181:6813","nonce":2848929566}]},"public_addr":"172.21.15.181:6809/2848929566","cluster_addr":"172.21.15.181:6811/2848929566","heartbeat_back_addr":"172.21.15.181:6815/2848929566","heartbeat_front_addr":"172.21.15.181:6813/2848929566","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.890007+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.992643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.981751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.973097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.899596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:58.014009+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.968644+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.910059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-06T18:38:08.373 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-09-06T18:38:08.373 INFO:tasks.ceph:Creating RBD pool 2023-09-06T18:38:08.374 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd pool create rbd 8 2023-09-06T18:38:09.334 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'rbd' created 2023-09-06T18:38:09.347 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-09-06T18:38:10.342 INFO:teuthology.orchestra.run.smithi099.stderr:enabled application 'rbd' on pool 'rbd' 2023-09-06T18:38:10.354 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-09-06T18:38:10.354 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-06T18:38:10.355 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-06T18:38:10.355 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-06T18:38:10.356 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-09-06T18:38:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:10.747 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":5,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2012733164},{"type":"v1","addr":"172.21.15.181:6833","nonce":2012733164}]},"active_addr":"172.21.15.181:6833/2012733164","active_change":"2023-09-06T18:37:56.233276+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4109,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2361109498}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":3437294617}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2565147394}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":4287692490}]}]} 2023-09-06T18:38:10.753 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-06T18:38:10.754 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-06T18:38:10.754 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-06T18:38:11.075 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:11.075 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":18,"fsid":"b0ee29a8-5d11-45f6-ae33-fae8cc67c8f5","created":"2023-09-06T18:37:50.238993+0000","modified":"2023-09-06T18:38:10.325184+0000","last_up_change":"2023-09-06T18:38:01.304703+0000","last_in_change":"2023-09-06T18:37:52.805129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-06T18:38:02.303794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-06T18:38:08.710309+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"60cafc8d-10aa-4647-8938-1cd7313d640c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2410739650},{"type":"v1","addr":"172.21.15.99:6801","nonce":2410739650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2410739650},{"type":"v1","addr":"172.21.15.99:6803","nonce":2410739650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2410739650},{"type":"v1","addr":"172.21.15.99:6807","nonce":2410739650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2410739650},{"type":"v1","addr":"172.21.15.99:6805","nonce":2410739650}]},"public_addr":"172.21.15.99:6801/2410739650","cluster_addr":"172.21.15.99:6803/2410739650","heartbeat_back_addr":"172.21.15.99:6807/2410739650","heartbeat_front_addr":"172.21.15.99:6805/2410739650","state":["exists","up"]},{"osd":1,"uuid":"d5358f90-6305-4897-870a-8d1a4f51ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1561643328},{"type":"v1","addr":"172.21.15.99:6809","nonce":1561643328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1561643328},{"type":"v1","addr":"172.21.15.99:6811","nonce":1561643328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1561643328},{"type":"v1","addr":"172.21.15.99:6815","nonce":1561643328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1561643328},{"type":"v1","addr":"172.21.15.99:6813","nonce":1561643328}]},"public_addr":"172.21.15.99:6809/1561643328","cluster_addr":"172.21.15.99:6811/1561643328","heartbeat_back_addr":"172.21.15.99:6815/1561643328","heartbeat_front_addr":"172.21.15.99:6813/1561643328","state":["exists","up"]},{"osd":2,"uuid":"edaaeed8-d9f2-4faf-aa0d-b4cf99422d69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1188692910},{"type":"v1","addr":"172.21.15.99:6819","nonce":1188692910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":1188692910},{"type":"v1","addr":"172.21.15.99:6823","nonce":1188692910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6829","nonce":1188692910},{"type":"v1","addr":"172.21.15.99:6831","nonce":1188692910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1188692910},{"type":"v1","addr":"172.21.15.99:6827","nonce":1188692910}]},"public_addr":"172.21.15.99:6819/1188692910","cluster_addr":"172.21.15.99:6823/1188692910","heartbeat_back_addr":"172.21.15.99:6831/1188692910","heartbeat_front_addr":"172.21.15.99:6827/1188692910","state":["exists","up"]},{"osd":3,"uuid":"81e09a9c-fb99-428e-8b78-c81e9362e7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2909409728},{"type":"v1","addr":"172.21.15.99:6818","nonce":2909409728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2909409728},{"type":"v1","addr":"172.21.15.99:6822","nonce":2909409728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2909409728},{"type":"v1","addr":"172.21.15.99:6830","nonce":2909409728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2909409728},{"type":"v1","addr":"172.21.15.99:6826","nonce":2909409728}]},"public_addr":"172.21.15.99:6818/2909409728","cluster_addr":"172.21.15.99:6822/2909409728","heartbeat_back_addr":"172.21.15.99:6830/2909409728","heartbeat_front_addr":"172.21.15.99:6826/2909409728","state":["exists","up"]},{"osd":4,"uuid":"9ebe4914-2849-4aa4-ba9a-ffd53485ca46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":520119356},{"type":"v1","addr":"172.21.15.181:6801","nonce":520119356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":520119356},{"type":"v1","addr":"172.21.15.181:6803","nonce":520119356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":520119356},{"type":"v1","addr":"172.21.15.181:6807","nonce":520119356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":520119356},{"type":"v1","addr":"172.21.15.181:6805","nonce":520119356}]},"public_addr":"172.21.15.181:6801/520119356","cluster_addr":"172.21.15.181:6803/520119356","heartbeat_back_addr":"172.21.15.181:6807/520119356","heartbeat_front_addr":"172.21.15.181:6805/520119356","state":["exists","up"]},{"osd":5,"uuid":"b0985b70-1d44-46b3-8e13-1f4ae4cf89b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3909493596},{"type":"v1","addr":"172.21.15.181:6817","nonce":3909493596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3909493596},{"type":"v1","addr":"172.21.15.181:6819","nonce":3909493596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3909493596},{"type":"v1","addr":"172.21.15.181:6823","nonce":3909493596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3909493596},{"type":"v1","addr":"172.21.15.181:6821","nonce":3909493596}]},"public_addr":"172.21.15.181:6817/3909493596","cluster_addr":"172.21.15.181:6819/3909493596","heartbeat_back_addr":"172.21.15.181:6823/3909493596","heartbeat_front_addr":"172.21.15.181:6821/3909493596","state":["exists","up"]},{"osd":6,"uuid":"7fe89e17-ee50-4832-827e-f770f2a97371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2835950561},{"type":"v1","addr":"172.21.15.181:6825","nonce":2835950561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2835950561},{"type":"v1","addr":"172.21.15.181:6827","nonce":2835950561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2835950561},{"type":"v1","addr":"172.21.15.181:6831","nonce":2835950561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2835950561},{"type":"v1","addr":"172.21.15.181:6829","nonce":2835950561}]},"public_addr":"172.21.15.181:6825/2835950561","cluster_addr":"172.21.15.181:6827/2835950561","heartbeat_back_addr":"172.21.15.181:6831/2835950561","heartbeat_front_addr":"172.21.15.181:6829/2835950561","state":["exists","up"]},{"osd":7,"uuid":"fb42cb6a-52fc-434a-b6cf-616d7fc470e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2848929566},{"type":"v1","addr":"172.21.15.181:6809","nonce":2848929566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2848929566},{"type":"v1","addr":"172.21.15.181:6811","nonce":2848929566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2848929566},{"type":"v1","addr":"172.21.15.181:6815","nonce":2848929566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2848929566},{"type":"v1","addr":"172.21.15.181:6813","nonce":2848929566}]},"public_addr":"172.21.15.181:6809/2848929566","cluster_addr":"172.21.15.181:6811/2848929566","heartbeat_back_addr":"172.21.15.181:6815/2848929566","heartbeat_front_addr":"172.21.15.181:6813/2848929566","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.890007+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.992643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.981751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.973097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.899596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:58.014009+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.968644+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.910059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-06T18:38:11.088 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-06T18:38:11.089 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-06T18:38:11.413 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:11.413 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":19,"fsid":"b0ee29a8-5d11-45f6-ae33-fae8cc67c8f5","created":"2023-09-06T18:37:50.238993+0000","modified":"2023-09-06T18:38:11.331389+0000","last_up_change":"2023-09-06T18:38:01.304703+0000","last_in_change":"2023-09-06T18:37:52.805129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-06T18:38:02.303794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-06T18:38:08.710309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"60cafc8d-10aa-4647-8938-1cd7313d640c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2410739650},{"type":"v1","addr":"172.21.15.99:6801","nonce":2410739650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2410739650},{"type":"v1","addr":"172.21.15.99:6803","nonce":2410739650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2410739650},{"type":"v1","addr":"172.21.15.99:6807","nonce":2410739650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2410739650},{"type":"v1","addr":"172.21.15.99:6805","nonce":2410739650}]},"public_addr":"172.21.15.99:6801/2410739650","cluster_addr":"172.21.15.99:6803/2410739650","heartbeat_back_addr":"172.21.15.99:6807/2410739650","heartbeat_front_addr":"172.21.15.99:6805/2410739650","state":["exists","up"]},{"osd":1,"uuid":"d5358f90-6305-4897-870a-8d1a4f51ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1561643328},{"type":"v1","addr":"172.21.15.99:6809","nonce":1561643328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1561643328},{"type":"v1","addr":"172.21.15.99:6811","nonce":1561643328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1561643328},{"type":"v1","addr":"172.21.15.99:6815","nonce":1561643328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1561643328},{"type":"v1","addr":"172.21.15.99:6813","nonce":1561643328}]},"public_addr":"172.21.15.99:6809/1561643328","cluster_addr":"172.21.15.99:6811/1561643328","heartbeat_back_addr":"172.21.15.99:6815/1561643328","heartbeat_front_addr":"172.21.15.99:6813/1561643328","state":["exists","up"]},{"osd":2,"uuid":"edaaeed8-d9f2-4faf-aa0d-b4cf99422d69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1188692910},{"type":"v1","addr":"172.21.15.99:6819","nonce":1188692910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":1188692910},{"type":"v1","addr":"172.21.15.99:6823","nonce":1188692910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6829","nonce":1188692910},{"type":"v1","addr":"172.21.15.99:6831","nonce":1188692910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1188692910},{"type":"v1","addr":"172.21.15.99:6827","nonce":1188692910}]},"public_addr":"172.21.15.99:6819/1188692910","cluster_addr":"172.21.15.99:6823/1188692910","heartbeat_back_addr":"172.21.15.99:6831/1188692910","heartbeat_front_addr":"172.21.15.99:6827/1188692910","state":["exists","up"]},{"osd":3,"uuid":"81e09a9c-fb99-428e-8b78-c81e9362e7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":2909409728},{"type":"v1","addr":"172.21.15.99:6818","nonce":2909409728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2909409728},{"type":"v1","addr":"172.21.15.99:6822","nonce":2909409728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2909409728},{"type":"v1","addr":"172.21.15.99:6830","nonce":2909409728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2909409728},{"type":"v1","addr":"172.21.15.99:6826","nonce":2909409728}]},"public_addr":"172.21.15.99:6818/2909409728","cluster_addr":"172.21.15.99:6822/2909409728","heartbeat_back_addr":"172.21.15.99:6830/2909409728","heartbeat_front_addr":"172.21.15.99:6826/2909409728","state":["exists","up"]},{"osd":4,"uuid":"9ebe4914-2849-4aa4-ba9a-ffd53485ca46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":520119356},{"type":"v1","addr":"172.21.15.181:6801","nonce":520119356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":520119356},{"type":"v1","addr":"172.21.15.181:6803","nonce":520119356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":520119356},{"type":"v1","addr":"172.21.15.181:6807","nonce":520119356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":520119356},{"type":"v1","addr":"172.21.15.181:6805","nonce":520119356}]},"public_addr":"172.21.15.181:6801/520119356","cluster_addr":"172.21.15.181:6803/520119356","heartbeat_back_addr":"172.21.15.181:6807/520119356","heartbeat_front_addr":"172.21.15.181:6805/520119356","state":["exists","up"]},{"osd":5,"uuid":"b0985b70-1d44-46b3-8e13-1f4ae4cf89b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3909493596},{"type":"v1","addr":"172.21.15.181:6817","nonce":3909493596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3909493596},{"type":"v1","addr":"172.21.15.181:6819","nonce":3909493596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3909493596},{"type":"v1","addr":"172.21.15.181:6823","nonce":3909493596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3909493596},{"type":"v1","addr":"172.21.15.181:6821","nonce":3909493596}]},"public_addr":"172.21.15.181:6817/3909493596","cluster_addr":"172.21.15.181:6819/3909493596","heartbeat_back_addr":"172.21.15.181:6823/3909493596","heartbeat_front_addr":"172.21.15.181:6821/3909493596","state":["exists","up"]},{"osd":6,"uuid":"7fe89e17-ee50-4832-827e-f770f2a97371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2835950561},{"type":"v1","addr":"172.21.15.181:6825","nonce":2835950561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2835950561},{"type":"v1","addr":"172.21.15.181:6827","nonce":2835950561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2835950561},{"type":"v1","addr":"172.21.15.181:6831","nonce":2835950561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2835950561},{"type":"v1","addr":"172.21.15.181:6829","nonce":2835950561}]},"public_addr":"172.21.15.181:6825/2835950561","cluster_addr":"172.21.15.181:6827/2835950561","heartbeat_back_addr":"172.21.15.181:6831/2835950561","heartbeat_front_addr":"172.21.15.181:6829/2835950561","state":["exists","up"]},{"osd":7,"uuid":"fb42cb6a-52fc-434a-b6cf-616d7fc470e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2848929566},{"type":"v1","addr":"172.21.15.181:6809","nonce":2848929566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2848929566},{"type":"v1","addr":"172.21.15.181:6811","nonce":2848929566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2848929566},{"type":"v1","addr":"172.21.15.181:6815","nonce":2848929566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2848929566},{"type":"v1","addr":"172.21.15.181:6813","nonce":2848929566}]},"public_addr":"172.21.15.181:6809/2848929566","cluster_addr":"172.21.15.181:6811/2848929566","heartbeat_back_addr":"172.21.15.181:6815/2848929566","heartbeat_front_addr":"172.21.15.181:6813/2848929566","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.890007+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.992643+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.981751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.973097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.899596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:58.014009+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.968644+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-06T18:37:57.910059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-06T18:38:11.427 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-06T18:38:11.427 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-06T18:38:11.428 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-06T18:38:11.429 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-06T18:38:11.429 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-09-06T18:38:11.430 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-09-06T18:38:11.431 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-09-06T18:38:11.431 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-09-06T18:38:11.722 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-09-06T18:38:11.722 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-09-06T18:38:11.723 INFO:teuthology.orchestra.run.smithi099.stdout:55834574855 2023-09-06T18:38:11.724 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-09-06T18:38:11.725 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-06T18:38:11.726 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-09-06T18:38:11.729 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-09-06T18:38:11.729 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-09-06T18:38:11.732 INFO:teuthology.orchestra.run.smithi099.stdout:55834574852 2023-09-06T18:38:11.734 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-09-06T18:38:11.740 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-09-06T18:38:11.740 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-06T18:38:11.741 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-09-06T18:38:11.742 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-06T18:38:11.745 INFO:teuthology.orchestra.run.smithi099.stdout:55834574855 2023-09-06T18:38:11.745 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-06T18:38:12.330 INFO:teuthology.orchestra.run.smithi099.stdout:55834574852 2023-09-06T18:38:12.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.7 2023-09-06T18:38:12.349 INFO:teuthology.orchestra.run.smithi099.stdout:55834574850 2023-09-06T18:38:12.356 INFO:teuthology.orchestra.run.smithi099.stdout:55834574852 2023-09-06T18:38:12.358 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2023-09-06T18:38:12.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.4 2023-09-06T18:38:12.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.5 2023-09-06T18:38:12.386 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-09-06T18:38:12.395 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-09-06T18:38:12.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.0 2023-09-06T18:38:12.404 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-09-06T18:38:12.404 DEBUG:teuthology.parallel:result is None 2023-09-06T18:38:12.408 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-09-06T18:38:12.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-09-06T18:38:12.412 DEBUG:teuthology.parallel:result is None 2023-09-06T18:38:12.423 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-09-06T18:38:12.423 DEBUG:teuthology.parallel:result is None 2023-09-06T18:38:12.442 INFO:teuthology.orchestra.run.smithi099.stdout:55834574855 2023-09-06T18:38:12.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-09-06T18:38:12.456 DEBUG:teuthology.parallel:result is None 2023-09-06T18:38:13.349 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-09-06T18:38:13.378 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-09-06T18:38:13.386 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-09-06T18:38:13.404 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-06T18:38:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-09-06T18:38:13.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-09-06T18:38:13.737 DEBUG:teuthology.parallel:result is None 2023-09-06T18:38:13.747 INFO:teuthology.orchestra.run.smithi099.stdout:55834574852 2023-09-06T18:38:13.753 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-09-06T18:38:13.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2023-09-06T18:38:13.763 DEBUG:teuthology.parallel:result is None 2023-09-06T18:38:13.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-09-06T18:38:13.768 DEBUG:teuthology.parallel:result is None 2023-09-06T18:38:13.795 INFO:teuthology.orchestra.run.smithi099.stdout:55834574855 2023-09-06T18:38:13.809 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-09-06T18:38:13.810 DEBUG:teuthology.parallel:result is None 2023-09-06T18:38:13.810 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-06T18:38:13.810 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-06T18:38:14.165 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:14.167 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-09-06T18:38:14.178 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-09-06T18:38:12.251053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218724,"kb_used_data":2532,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755996,"statfs":{"total":773094113280,"available":772870139904,"internally_reserved":0,"allocated":2592768,"data_stored":1621639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000848"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-06T18:38:10.349979+0000","last_change":"2023-09-06T18:38:10.347780+0000","last_active":"2023-09-06T18:38:10.349979+0000","last_peered":"2023-09-06T18:38:10.349979+0000","last_clean":"2023-09-06T18:38:10.349979+0000","last_became_active":"2023-09-06T18:38:10.347466+0000","last_became_peered":"2023-09-06T18:38:10.347466+0000","last_unstale":"2023-09-06T18:38:10.349979+0000","last_undegraded":"2023-09-06T18:38:10.349979+0000","last_fullsized":"2023-09-06T18:38:10.349979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-08T02:34:40.918205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-06T18:38:10.339739+0000","last_change":"2023-09-06T18:38:10.339150+0000","last_active":"2023-09-06T18:38:10.339739+0000","last_peered":"2023-09-06T18:38:10.339739+0000","last_clean":"2023-09-06T18:38:10.339739+0000","last_became_active":"2023-09-06T18:38:10.338856+0000","last_became_peered":"2023-09-06T18:38:10.338856+0000","last_unstale":"2023-09-06T18:38:10.339739+0000","last_undegraded":"2023-09-06T18:38:10.339739+0000","last_fullsized":"2023-09-06T18:38:10.339739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-08T00:34:27.986193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-06T18:38:11.340741+0000","last_change":"2023-09-06T18:38:10.718409+0000","last_active":"2023-09-06T18:38:11.340741+0000","last_peered":"2023-09-06T18:38:11.340741+0000","last_clean":"2023-09-06T18:38:11.340741+0000","last_became_active":"2023-09-06T18:38:10.718182+0000","last_became_peered":"2023-09-06T18:38:10.718182+0000","last_unstale":"2023-09-06T18:38:11.340741+0000","last_undegraded":"2023-09-06T18:38:11.340741+0000","last_fullsized":"2023-09-06T18:38:11.340741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-07T20:23:37.240865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-06T18:38:11.340721+0000","last_change":"2023-09-06T18:38:10.345154+0000","last_active":"2023-09-06T18:38:11.340721+0000","last_peered":"2023-09-06T18:38:11.340721+0000","last_clean":"2023-09-06T18:38:11.340721+0000","last_became_active":"2023-09-06T18:38:10.344885+0000","last_became_peered":"2023-09-06T18:38:10.344885+0000","last_unstale":"2023-09-06T18:38:11.340721+0000","last_undegraded":"2023-09-06T18:38:11.340721+0000","last_fullsized":"2023-09-06T18:38:11.340721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-08T02:44:59.120546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-06T18:38:10.341884+0000","last_change":"2023-09-06T18:38:03.778242+0000","last_active":"2023-09-06T18:38:10.341884+0000","last_peered":"2023-09-06T18:38:10.341884+0000","last_clean":"2023-09-06T18:38:10.341884+0000","last_became_active":"2023-09-06T18:38:03.777061+0000","last_became_peered":"2023-09-06T18:38:03.777061+0000","last_unstale":"2023-09-06T18:38:10.341884+0000","last_undegraded":"2023-09-06T18:38:10.341884+0000","last_fullsized":"2023-09-06T18:38:10.341884+0000","mapping_epoch":14,"log_start":"15'100","ondisk_log_start":"15'100","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:02.520878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:02.520878+0000","last_clean_scrub_stamp":"2023-09-06T18:38:02.520878+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-08T06:20:37.153826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-06T18:38:11.373643+0000","last_change":"2023-09-06T18:38:10.341584+0000","last_active":"2023-09-06T18:38:11.373643+0000","last_peered":"2023-09-06T18:38:11.373643+0000","last_clean":"2023-09-06T18:38:11.373643+0000","last_became_active":"2023-09-06T18:38:10.341273+0000","last_became_peered":"2023-09-06T18:38:10.341273+0000","last_unstale":"2023-09-06T18:38:11.373643+0000","last_undegraded":"2023-09-06T18:38:11.373643+0000","last_fullsized":"2023-09-06T18:38:11.373643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-08T02:44:59.120546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-06T18:38:10.342388+0000","last_change":"2023-09-06T18:38:10.341473+0000","last_active":"2023-09-06T18:38:10.342388+0000","last_peered":"2023-09-06T18:38:10.342388+0000","last_clean":"2023-09-06T18:38:10.342388+0000","last_became_active":"2023-09-06T18:38:10.341155+0000","last_became_peered":"2023-09-06T18:38:10.341155+0000","last_unstale":"2023-09-06T18:38:10.342388+0000","last_undegraded":"2023-09-06T18:38:10.342388+0000","last_fullsized":"2023-09-06T18:38:10.342388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-08T03:54:00.664951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-06T18:38:11.373641+0000","last_change":"2023-09-06T18:38:10.339206+0000","last_active":"2023-09-06T18:38:11.373641+0000","last_peered":"2023-09-06T18:38:11.373641+0000","last_clean":"2023-09-06T18:38:11.373641+0000","last_became_active":"2023-09-06T18:38:10.338900+0000","last_became_peered":"2023-09-06T18:38:10.338900+0000","last_unstale":"2023-09-06T18:38:11.373641+0000","last_undegraded":"2023-09-06T18:38:11.373641+0000","last_fullsized":"2023-09-06T18:38:11.373641+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-07T20:23:37.240865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-06T18:38:10.722060+0000","last_change":"2023-09-06T18:38:10.720934+0000","last_active":"2023-09-06T18:38:10.722060+0000","last_peered":"2023-09-06T18:38:10.722060+0000","last_clean":"2023-09-06T18:38:10.722060+0000","last_became_active":"2023-09-06T18:38:10.720620+0000","last_became_peered":"2023-09-06T18:38:10.720620+0000","last_unstale":"2023-09-06T18:38:10.722060+0000","last_undegraded":"2023-09-06T18:38:10.722060+0000","last_fullsized":"2023-09-06T18:38:10.722060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-08T03:54:00.664951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":647648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-06T18:38:14.180 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-06T18:38:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:14.503 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-09-06T18:38:14.514 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-09-06T18:38:14.251461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218724,"kb_used_data":2532,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755996,"statfs":{"total":773094113280,"available":772870139904,"internally_reserved":0,"allocated":2592768,"data_stored":1621639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001257"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-06T18:38:10.349979+0000","last_change":"2023-09-06T18:38:10.347780+0000","last_active":"2023-09-06T18:38:10.349979+0000","last_peered":"2023-09-06T18:38:10.349979+0000","last_clean":"2023-09-06T18:38:10.349979+0000","last_became_active":"2023-09-06T18:38:10.347466+0000","last_became_peered":"2023-09-06T18:38:10.347466+0000","last_unstale":"2023-09-06T18:38:10.349979+0000","last_undegraded":"2023-09-06T18:38:10.349979+0000","last_fullsized":"2023-09-06T18:38:10.349979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-08T02:34:40.918205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-06T18:38:10.339739+0000","last_change":"2023-09-06T18:38:10.339150+0000","last_active":"2023-09-06T18:38:10.339739+0000","last_peered":"2023-09-06T18:38:10.339739+0000","last_clean":"2023-09-06T18:38:10.339739+0000","last_became_active":"2023-09-06T18:38:10.338856+0000","last_became_peered":"2023-09-06T18:38:10.338856+0000","last_unstale":"2023-09-06T18:38:10.339739+0000","last_undegraded":"2023-09-06T18:38:10.339739+0000","last_fullsized":"2023-09-06T18:38:10.339739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-08T00:34:27.986193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-06T18:38:11.340741+0000","last_change":"2023-09-06T18:38:10.718409+0000","last_active":"2023-09-06T18:38:11.340741+0000","last_peered":"2023-09-06T18:38:11.340741+0000","last_clean":"2023-09-06T18:38:11.340741+0000","last_became_active":"2023-09-06T18:38:10.718182+0000","last_became_peered":"2023-09-06T18:38:10.718182+0000","last_unstale":"2023-09-06T18:38:11.340741+0000","last_undegraded":"2023-09-06T18:38:11.340741+0000","last_fullsized":"2023-09-06T18:38:11.340741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-07T20:23:37.240865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-06T18:38:11.340721+0000","last_change":"2023-09-06T18:38:10.345154+0000","last_active":"2023-09-06T18:38:11.340721+0000","last_peered":"2023-09-06T18:38:11.340721+0000","last_clean":"2023-09-06T18:38:11.340721+0000","last_became_active":"2023-09-06T18:38:10.344885+0000","last_became_peered":"2023-09-06T18:38:10.344885+0000","last_unstale":"2023-09-06T18:38:11.340721+0000","last_undegraded":"2023-09-06T18:38:11.340721+0000","last_fullsized":"2023-09-06T18:38:11.340721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-08T02:44:59.120546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-06T18:38:10.341884+0000","last_change":"2023-09-06T18:38:03.778242+0000","last_active":"2023-09-06T18:38:10.341884+0000","last_peered":"2023-09-06T18:38:10.341884+0000","last_clean":"2023-09-06T18:38:10.341884+0000","last_became_active":"2023-09-06T18:38:03.777061+0000","last_became_peered":"2023-09-06T18:38:03.777061+0000","last_unstale":"2023-09-06T18:38:10.341884+0000","last_undegraded":"2023-09-06T18:38:10.341884+0000","last_fullsized":"2023-09-06T18:38:10.341884+0000","mapping_epoch":14,"log_start":"15'100","ondisk_log_start":"15'100","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:02.520878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:02.520878+0000","last_clean_scrub_stamp":"2023-09-06T18:38:02.520878+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-08T06:20:37.153826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-06T18:38:11.373643+0000","last_change":"2023-09-06T18:38:10.341584+0000","last_active":"2023-09-06T18:38:11.373643+0000","last_peered":"2023-09-06T18:38:11.373643+0000","last_clean":"2023-09-06T18:38:11.373643+0000","last_became_active":"2023-09-06T18:38:10.341273+0000","last_became_peered":"2023-09-06T18:38:10.341273+0000","last_unstale":"2023-09-06T18:38:11.373643+0000","last_undegraded":"2023-09-06T18:38:11.373643+0000","last_fullsized":"2023-09-06T18:38:11.373643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-08T02:44:59.120546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-06T18:38:10.342388+0000","last_change":"2023-09-06T18:38:10.341473+0000","last_active":"2023-09-06T18:38:10.342388+0000","last_peered":"2023-09-06T18:38:10.342388+0000","last_clean":"2023-09-06T18:38:10.342388+0000","last_became_active":"2023-09-06T18:38:10.341155+0000","last_became_peered":"2023-09-06T18:38:10.341155+0000","last_unstale":"2023-09-06T18:38:10.342388+0000","last_undegraded":"2023-09-06T18:38:10.342388+0000","last_fullsized":"2023-09-06T18:38:10.342388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-08T03:54:00.664951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-06T18:38:11.373641+0000","last_change":"2023-09-06T18:38:10.339206+0000","last_active":"2023-09-06T18:38:11.373641+0000","last_peered":"2023-09-06T18:38:11.373641+0000","last_clean":"2023-09-06T18:38:11.373641+0000","last_became_active":"2023-09-06T18:38:10.338900+0000","last_became_peered":"2023-09-06T18:38:10.338900+0000","last_unstale":"2023-09-06T18:38:11.373641+0000","last_undegraded":"2023-09-06T18:38:11.373641+0000","last_fullsized":"2023-09-06T18:38:11.373641+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-07T20:23:37.240865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-06T18:38:10.722060+0000","last_change":"2023-09-06T18:38:10.720934+0000","last_active":"2023-09-06T18:38:10.722060+0000","last_peered":"2023-09-06T18:38:10.722060+0000","last_clean":"2023-09-06T18:38:10.722060+0000","last_became_active":"2023-09-06T18:38:10.720620+0000","last_became_peered":"2023-09-06T18:38:10.720620+0000","last_unstale":"2023-09-06T18:38:10.722060+0000","last_undegraded":"2023-09-06T18:38:10.722060+0000","last_fullsized":"2023-09-06T18:38:10.722060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-06T18:38:09.319965+0000","last_clean_scrub_stamp":"2023-09-06T18:38:09.319965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-08T03:54:00.664951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":647648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-06T18:38:14.516 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-06T18:38:14.516 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-06T18:38:14.516 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-09-06T18:38:14.517 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-06T18:38:14.876 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:14.876 INFO:teuthology.orchestra.run.smithi099.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-06T18:38:14.890 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-09-06T18:38:14.891 INFO:teuthology.run_tasks:Running task openssl_keys... 2023-09-06T18:38:14.905 INFO:teuthology.run_tasks:Running task rgw... 2023-09-06T18:38:14.919 DEBUG:tasks.rgw:config is {'client.0': None} 2023-09-06T18:38:14.920 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2023-09-06T18:38:14.920 INFO:tasks.rgw:Creating data pools 2023-09-06T18:38:14.920 DEBUG:tasks.rgw:Obtaining remote for client client.0 2023-09-06T18:38:14.921 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster ceph 2023-09-06T18:38:16.011 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'default.rgw.buckets.data' created 2023-09-06T18:38:16.025 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2023-09-06T18:38:17.026 INFO:teuthology.orchestra.run.smithi099.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2023-09-06T18:38:17.038 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2023-09-06T18:38:18.027 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'default.rgw.buckets.index' created 2023-09-06T18:38:18.039 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2023-09-06T18:38:19.039 INFO:teuthology.orchestra.run.smithi099.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2023-09-06T18:38:19.050 DEBUG:tasks.rgw:Pools created 2023-09-06T18:38:19.051 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2023-09-06T18:38:19.051 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2023-09-06T18:38:19.052 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2023-09-06T18:38:19.141 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setuser ceph since I am not root 2023-09-06T18:38:19.141 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setgroup ceph since I am not root 2023-09-06T18:38:19.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:19.148+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.038+0000 7f2173d11780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:38:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.038+0000 7f2173d11780 20 realm 2023-09-06T18:38:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.038+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.039+0000 7f2173d11780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:38:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.039+0000 7f2173d11780 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-09-06T18:38:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.039+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.039+0000 7f2173d11780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:38:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.039+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:21.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.043+0000 7f2173d11780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:38:21.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.043+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:21.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.044+0000 7f2173d11780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:38:21.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.044+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:21.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.045+0000 7f2173d11780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:38:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.045+0000 7f2173d11780 10 Creating default zonegroup 2023-09-06T18:38:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.045+0000 7f2173d11780 10 couldn't find old data placement pools config, setting up new ones for the zone 2023-09-06T18:38:21.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.058+0000 7f2173d11780 10 WARNING: driver->list_zones() returned r=-2 2023-09-06T18:38:21.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.058+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:21.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.058+0000 7f2173d11780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:38:21.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.063+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:21.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.064+0000 7f2173d11780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:38:21.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.064+0000 7f2173d11780 10 could not read realm id: (2) No such file or directory 2023-09-06T18:38:21.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.064+0000 7f2173d11780 10 WARNING: failed to set zone as default, r=-22 2023-09-06T18:38:21.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.064+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:21.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.064+0000 7f2173d11780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:38:21.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.067+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:21.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.068+0000 7f2173d11780 20 rados_obj.operate() r=0 bl.length=874 2023-09-06T18:38:21.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.068+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:21.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.068+0000 7f2173d11780 20 rados_obj.operate() r=0 bl.length=420 2023-09-06T18:38:21.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.068+0000 7f2173d11780 10 Using default name default 2023-09-06T18:38:21.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.068+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:21.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.069+0000 7f2173d11780 20 rados_obj.operate() r=0 bl.length=46 2023-09-06T18:38:21.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.069+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:21.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.069+0000 7f2173d11780 20 rados_obj.operate() r=0 bl.length=874 2023-09-06T18:38:21.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.069+0000 7f2173d11780 20 zone default found 2023-09-06T18:38:21.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.069+0000 7f2173d11780 4 Realm: () 2023-09-06T18:38:21.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.069+0000 7f2173d11780 4 ZoneGroup: default (aafaa245-59dc-4ab8-adab-a5293aa3a8a6) 2023-09-06T18:38:21.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.069+0000 7f2173d11780 4 Zone: default (d19ec83f-a74c-4fd7-8a2d-747094fce3d9) 2023-09-06T18:38:21.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.069+0000 7f2173d11780 10 cannot find current period zonegroup using local zonegroup configuration 2023-09-06T18:38:21.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.069+0000 7f2173d11780 20 zonegroup default 2023-09-06T18:38:21.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.069+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:21.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.069+0000 7f2173d11780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:38:21.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:21.069+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:23.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:23.051+0000 7f2173d11780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:38:23.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:23.051+0000 7f2173d11780 20 rados->read ofs=0 len=0 2023-09-06T18:38:23.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:23.051+0000 7f2173d11780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:38:23.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:23.051+0000 7f2173d11780 20 started sync module instance, tier type = 2023-09-06T18:38:23.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:23.051+0000 7f2173d11780 20 started zone id=d19ec83f-a74c-4fd7-8a2d-747094fce3d9 (name=default) with tier type = 2023-09-06T18:38:25.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.078+0000 7f2173d11780 20 add_watcher() i=0 2023-09-06T18:38:25.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.078+0000 7f2173d11780 20 add_watcher() i=1 2023-09-06T18:38:25.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.078+0000 7f2173d11780 20 add_watcher() i=2 2023-09-06T18:38:25.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.078+0000 7f2173d11780 20 add_watcher() i=3 2023-09-06T18:38:25.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.078+0000 7f2173d11780 20 add_watcher() i=4 2023-09-06T18:38:25.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.078+0000 7f2173d11780 20 add_watcher() i=5 2023-09-06T18:38:25.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.080+0000 7f2173d11780 20 add_watcher() i=6 2023-09-06T18:38:25.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.083+0000 7f2173d11780 20 add_watcher() i=7 2023-09-06T18:38:25.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.083+0000 7f2173d11780 2 all 8 watchers are set, enabling cache 2023-09-06T18:38:25.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.083+0000 7f2173d11780 5 tl::expected, obj_version>, boost::system::error_code> logback_generations::read(const DoutPrefixProvider*, optional_yield):410: oid=data_loggenerations_metadata not found 2023-09-06T18:38:25.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.083+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.0 2023-09-06T18:38:25.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.083+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.084+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.0 does not exist 2023-09-06T18:38:25.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.084+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.1 2023-09-06T18:38:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.084+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.084+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.1 does not exist 2023-09-06T18:38:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.084+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.2 2023-09-06T18:38:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.084+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.084+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.2 does not exist 2023-09-06T18:38:25.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.084+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.3 2023-09-06T18:38:25.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.084+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.084+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.3 does not exist 2023-09-06T18:38:25.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.084+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.4 2023-09-06T18:38:25.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.084+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.085+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.4 does not exist 2023-09-06T18:38:25.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.085+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.5 2023-09-06T18:38:25.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.085+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.085+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.5 does not exist 2023-09-06T18:38:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.085+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.6 2023-09-06T18:38:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.085+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.085+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.6 does not exist 2023-09-06T18:38:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.085+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.7 2023-09-06T18:38:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.085+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.085+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.7 does not exist 2023-09-06T18:38:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.085+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.8 2023-09-06T18:38:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.085+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.086+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.8 does not exist 2023-09-06T18:38:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.086+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.9 2023-09-06T18:38:25.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.086+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.086+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.9 does not exist 2023-09-06T18:38:25.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.086+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.10 2023-09-06T18:38:25.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.086+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.086+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.10 does not exist 2023-09-06T18:38:25.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.086+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.11 2023-09-06T18:38:25.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.086+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.087+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.11 does not exist 2023-09-06T18:38:25.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.087+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.12 2023-09-06T18:38:25.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.087+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.087+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.12 does not exist 2023-09-06T18:38:25.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.087+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.13 2023-09-06T18:38:25.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.087+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.087+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.13 does not exist 2023-09-06T18:38:25.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.087+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.14 2023-09-06T18:38:25.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.087+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.088+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.14 does not exist 2023-09-06T18:38:25.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.088+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.15 2023-09-06T18:38:25.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.088+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.088+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.15 does not exist 2023-09-06T18:38:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.088+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.16 2023-09-06T18:38:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.088+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.088+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.16 does not exist 2023-09-06T18:38:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.088+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.17 2023-09-06T18:38:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.088+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.088+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.17 does not exist 2023-09-06T18:38:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.088+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.18 2023-09-06T18:38:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.088+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.089+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.18 does not exist 2023-09-06T18:38:25.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.089+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.19 2023-09-06T18:38:25.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.089+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.089+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.19 does not exist 2023-09-06T18:38:25.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.089+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.20 2023-09-06T18:38:25.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.089+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.089+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.20 does not exist 2023-09-06T18:38:25.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.089+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.21 2023-09-06T18:38:25.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.089+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.089+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.21 does not exist 2023-09-06T18:38:25.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.089+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.22 2023-09-06T18:38:25.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.089+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.090+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.22 does not exist 2023-09-06T18:38:25.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.090+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.23 2023-09-06T18:38:25.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.090+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.090+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.23 does not exist 2023-09-06T18:38:25.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.090+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.24 2023-09-06T18:38:25.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.090+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.090+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.24 does not exist 2023-09-06T18:38:25.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.090+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.25 2023-09-06T18:38:25.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.090+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.090+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.25 does not exist 2023-09-06T18:38:25.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.090+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.26 2023-09-06T18:38:25.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.090+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.091+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.26 does not exist 2023-09-06T18:38:25.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.091+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.27 2023-09-06T18:38:25.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.091+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.091+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.27 does not exist 2023-09-06T18:38:25.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.091+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.28 2023-09-06T18:38:25.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.091+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.091+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.28 does not exist 2023-09-06T18:38:25.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.091+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.29 2023-09-06T18:38:25.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.091+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.091+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.29 does not exist 2023-09-06T18:38:25.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.091+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.30 2023-09-06T18:38:25.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.091+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.092+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.30 does not exist 2023-09-06T18:38:25.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.092+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.31 2023-09-06T18:38:25.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.092+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.092+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.31 does not exist 2023-09-06T18:38:25.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.092+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.32 2023-09-06T18:38:25.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.092+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.092+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.32 does not exist 2023-09-06T18:38:25.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.092+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.33 2023-09-06T18:38:25.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.092+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.092+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.33 does not exist 2023-09-06T18:38:25.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.092+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.34 2023-09-06T18:38:25.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.092+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.093+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.34 does not exist 2023-09-06T18:38:25.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.093+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.35 2023-09-06T18:38:25.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.093+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.093+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.35 does not exist 2023-09-06T18:38:25.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.093+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.36 2023-09-06T18:38:25.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.093+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.093+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.36 does not exist 2023-09-06T18:38:25.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.093+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.37 2023-09-06T18:38:25.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.093+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.093+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.37 does not exist 2023-09-06T18:38:25.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.093+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.38 2023-09-06T18:38:25.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.093+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.094+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.38 does not exist 2023-09-06T18:38:25.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.094+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.39 2023-09-06T18:38:25.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.094+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.094+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.39 does not exist 2023-09-06T18:38:25.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.094+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.40 2023-09-06T18:38:25.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.094+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.094+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.40 does not exist 2023-09-06T18:38:25.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.094+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.41 2023-09-06T18:38:25.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.094+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.094+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.41 does not exist 2023-09-06T18:38:25.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.094+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.42 2023-09-06T18:38:25.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.094+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.095+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.42 does not exist 2023-09-06T18:38:25.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.095+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.43 2023-09-06T18:38:25.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.095+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.095+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.43 does not exist 2023-09-06T18:38:25.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.095+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.44 2023-09-06T18:38:25.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.095+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.095+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.44 does not exist 2023-09-06T18:38:25.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.095+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.45 2023-09-06T18:38:25.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.095+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.096+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.45 does not exist 2023-09-06T18:38:25.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.096+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.46 2023-09-06T18:38:25.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.096+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.096+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.46 does not exist 2023-09-06T18:38:25.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.096+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.47 2023-09-06T18:38:25.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.096+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.097+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.47 does not exist 2023-09-06T18:38:25.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.097+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.48 2023-09-06T18:38:25.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.097+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.097+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.48 does not exist 2023-09-06T18:38:25.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.097+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.49 2023-09-06T18:38:25.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.097+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.097+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.49 does not exist 2023-09-06T18:38:25.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.097+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.50 2023-09-06T18:38:25.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.097+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.097+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.50 does not exist 2023-09-06T18:38:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.097+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.51 2023-09-06T18:38:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.097+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.098+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.51 does not exist 2023-09-06T18:38:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.098+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.52 2023-09-06T18:38:25.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.098+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.098+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.52 does not exist 2023-09-06T18:38:25.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.098+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.53 2023-09-06T18:38:25.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.098+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.098+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.53 does not exist 2023-09-06T18:38:25.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.098+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.54 2023-09-06T18:38:25.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.098+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.098+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.54 does not exist 2023-09-06T18:38:25.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.098+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.55 2023-09-06T18:38:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.098+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.099+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.55 does not exist 2023-09-06T18:38:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.099+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.56 2023-09-06T18:38:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.099+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.099+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.56 does not exist 2023-09-06T18:38:25.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.099+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.57 2023-09-06T18:38:25.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.099+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.099+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.57 does not exist 2023-09-06T18:38:25.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.099+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.58 2023-09-06T18:38:25.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.099+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.099+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.58 does not exist 2023-09-06T18:38:25.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.099+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.59 2023-09-06T18:38:25.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.099+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.100+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.59 does not exist 2023-09-06T18:38:25.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.100+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.60 2023-09-06T18:38:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.100+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.100+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.60 does not exist 2023-09-06T18:38:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.100+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.61 2023-09-06T18:38:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.100+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.100+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.61 does not exist 2023-09-06T18:38:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.100+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.62 2023-09-06T18:38:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.100+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.101+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.62 does not exist 2023-09-06T18:38:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.101+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.63 2023-09-06T18:38:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.101+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.101+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.63 does not exist 2023-09-06T18:38:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.101+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.64 2023-09-06T18:38:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.101+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.101+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.64 does not exist 2023-09-06T18:38:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.101+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.65 2023-09-06T18:38:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.101+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.101+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.65 does not exist 2023-09-06T18:38:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.101+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.66 2023-09-06T18:38:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.101+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.66 does not exist 2023-09-06T18:38:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.67 2023-09-06T18:38:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.67 does not exist 2023-09-06T18:38:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.68 2023-09-06T18:38:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.68 does not exist 2023-09-06T18:38:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.69 2023-09-06T18:38:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.69 does not exist 2023-09-06T18:38:25.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.70 2023-09-06T18:38:25.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.70 does not exist 2023-09-06T18:38:25.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.71 2023-09-06T18:38:25.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.102+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.103+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.71 does not exist 2023-09-06T18:38:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.103+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.72 2023-09-06T18:38:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.103+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.103+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.72 does not exist 2023-09-06T18:38:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.103+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.73 2023-09-06T18:38:25.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.103+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.103+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.73 does not exist 2023-09-06T18:38:25.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.103+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.74 2023-09-06T18:38:25.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.103+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.103+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.74 does not exist 2023-09-06T18:38:25.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.103+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.75 2023-09-06T18:38:25.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.103+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.104+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.75 does not exist 2023-09-06T18:38:25.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.104+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.76 2023-09-06T18:38:25.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.104+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.104+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.76 does not exist 2023-09-06T18:38:25.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.104+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.77 2023-09-06T18:38:25.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.104+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.104+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.77 does not exist 2023-09-06T18:38:25.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.104+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.78 2023-09-06T18:38:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.104+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.104+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.78 does not exist 2023-09-06T18:38:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.104+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.79 2023-09-06T18:38:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.104+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.105+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.79 does not exist 2023-09-06T18:38:25.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.105+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.80 2023-09-06T18:38:25.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.105+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.105+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.80 does not exist 2023-09-06T18:38:25.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.105+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.81 2023-09-06T18:38:25.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.105+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.105+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.81 does not exist 2023-09-06T18:38:25.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.105+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.82 2023-09-06T18:38:25.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.105+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.105+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.82 does not exist 2023-09-06T18:38:25.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.105+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.83 2023-09-06T18:38:25.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.105+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.106+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.83 does not exist 2023-09-06T18:38:25.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.106+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.84 2023-09-06T18:38:25.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.106+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.106+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.84 does not exist 2023-09-06T18:38:25.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.106+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.85 2023-09-06T18:38:25.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.106+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.106+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.85 does not exist 2023-09-06T18:38:25.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.106+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.86 2023-09-06T18:38:25.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.106+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.106+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.86 does not exist 2023-09-06T18:38:25.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.106+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.87 2023-09-06T18:38:25.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.106+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.107+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.87 does not exist 2023-09-06T18:38:25.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.107+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.88 2023-09-06T18:38:25.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.107+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.107+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.88 does not exist 2023-09-06T18:38:25.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.107+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.89 2023-09-06T18:38:25.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.107+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.107+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.89 does not exist 2023-09-06T18:38:25.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.107+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.90 2023-09-06T18:38:25.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.107+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.107+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.90 does not exist 2023-09-06T18:38:25.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.107+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.91 2023-09-06T18:38:25.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.107+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.108+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.91 does not exist 2023-09-06T18:38:25.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.108+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.92 2023-09-06T18:38:25.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.108+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.108+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.92 does not exist 2023-09-06T18:38:25.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.108+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.93 2023-09-06T18:38:25.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.108+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.108+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.93 does not exist 2023-09-06T18:38:25.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.108+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.94 2023-09-06T18:38:25.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.108+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.108+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.94 does not exist 2023-09-06T18:38:25.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.108+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.95 2023-09-06T18:38:25.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.108+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.109+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.95 does not exist 2023-09-06T18:38:25.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.109+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.96 2023-09-06T18:38:25.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.109+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.109+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.96 does not exist 2023-09-06T18:38:25.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.109+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.97 2023-09-06T18:38:25.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.109+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.109+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.97 does not exist 2023-09-06T18:38:25.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.109+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.98 2023-09-06T18:38:25.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.109+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.109+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.98 does not exist 2023-09-06T18:38:25.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.109+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.99 2023-09-06T18:38:25.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.109+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.110+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.99 does not exist 2023-09-06T18:38:25.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.110+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.100 2023-09-06T18:38:25.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.110+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.110+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.100 does not exist 2023-09-06T18:38:25.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.110+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.101 2023-09-06T18:38:25.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.110+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.110+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.101 does not exist 2023-09-06T18:38:25.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.110+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.102 2023-09-06T18:38:25.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.110+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.110+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.102 does not exist 2023-09-06T18:38:25.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.110+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.103 2023-09-06T18:38:25.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.110+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.103 does not exist 2023-09-06T18:38:25.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.104 2023-09-06T18:38:25.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.104 does not exist 2023-09-06T18:38:25.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.105 2023-09-06T18:38:25.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.105 does not exist 2023-09-06T18:38:25.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.106 2023-09-06T18:38:25.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.106 does not exist 2023-09-06T18:38:25.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.107 2023-09-06T18:38:25.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.107 does not exist 2023-09-06T18:38:25.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.108 2023-09-06T18:38:25.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.111+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.112+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.108 does not exist 2023-09-06T18:38:25.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.112+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.109 2023-09-06T18:38:25.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.112+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.112+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.109 does not exist 2023-09-06T18:38:25.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.112+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.110 2023-09-06T18:38:25.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.112+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.112+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.110 does not exist 2023-09-06T18:38:25.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.112+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.111 2023-09-06T18:38:25.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.112+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.112+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.111 does not exist 2023-09-06T18:38:25.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.112+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.112 2023-09-06T18:38:25.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.112+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.112 does not exist 2023-09-06T18:38:25.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.113 2023-09-06T18:38:25.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.113 does not exist 2023-09-06T18:38:25.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.114 2023-09-06T18:38:25.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.114 does not exist 2023-09-06T18:38:25.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.115 2023-09-06T18:38:25.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.115 does not exist 2023-09-06T18:38:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.116 2023-09-06T18:38:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.116 does not exist 2023-09-06T18:38:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.117 2023-09-06T18:38:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.113+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.117 does not exist 2023-09-06T18:38:25.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.118 2023-09-06T18:38:25.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.118 does not exist 2023-09-06T18:38:25.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.119 2023-09-06T18:38:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.119 does not exist 2023-09-06T18:38:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.120 2023-09-06T18:38:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.120 does not exist 2023-09-06T18:38:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.121 2023-09-06T18:38:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.121 does not exist 2023-09-06T18:38:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.122 2023-09-06T18:38:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.114+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.115+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.122 does not exist 2023-09-06T18:38:25.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.115+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.123 2023-09-06T18:38:25.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.115+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.115+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.123 does not exist 2023-09-06T18:38:25.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.115+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.124 2023-09-06T18:38:25.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.115+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.116+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.124 does not exist 2023-09-06T18:38:25.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.116+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.125 2023-09-06T18:38:25.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.116+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.116+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.125 does not exist 2023-09-06T18:38:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.116+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.126 2023-09-06T18:38:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.116+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.116+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.126 does not exist 2023-09-06T18:38:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.116+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.127 2023-09-06T18:38:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.116+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.116+0000 7f2173d11780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.127 does not exist 2023-09-06T18:38:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.116+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::create(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, std::optional >, bool, uint64_t, uint64_t):1210 entering 2023-09-06T18:38:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.121+0000 7f2173d11780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-09-06T18:38:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.125+0000 7f2158c8d700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-09-06T18:38:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.125+0000 7f2173d11780 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-09-06T18:38:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:25.125+0000 7f2173d11780 20 check_secure_mon_conn(): mode 1 is insecure 2023-09-06T18:38:28.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.050+0000 7f2173d11780 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2023-09-06T18:38:28.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.050+0000 7f2173d11780 5 note: GC not initialized 2023-09-06T18:38:28.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.051+0000 7f2154484700 20 reqs_thread_entry: start 2023-09-06T18:38:28.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.051+0000 7f2173d11780 20 init_complete bucket index max shards: 11 2023-09-06T18:38:28.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.051+0000 7f2152480700 20 reqs_thread_entry: start 2023-09-06T18:38:28.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.084+0000 7f2158c8d700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-09-06T18:38:28.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.088+0000 7f2173d11780 20 remove_watcher() i=0 2023-09-06T18:38:28.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.088+0000 7f2173d11780 2 removed watcher, disabling cache 2023-09-06T18:38:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.089+0000 7f2173d11780 20 remove_watcher() i=1 2023-09-06T18:38:28.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.092+0000 7f2173d11780 20 remove_watcher() i=2 2023-09-06T18:38:28.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.096+0000 7f2173d11780 20 remove_watcher() i=3 2023-09-06T18:38:28.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.097+0000 7f2173d11780 20 remove_watcher() i=4 2023-09-06T18:38:28.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.100+0000 7f2173d11780 20 remove_watcher() i=5 2023-09-06T18:38:28.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.101+0000 7f2173d11780 20 remove_watcher() i=6 2023-09-06T18:38:28.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.102+0000 7f2173d11780 20 remove_watcher() i=7 2023-09-06T18:38:28.107 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2023-09-06T18:38:28.108 DEBUG:tasks.util.rgw: json result: [] 2023-09-06T18:38:28.108 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2023-09-06T18:38:28.109 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2023-09-06T18:38:28.109 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2023-09-06T18:38:28.109 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2023-09-06T18:38:28.161 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setuser ceph since I am not root 2023-09-06T18:38:28.161 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setgroup ceph since I am not root 2023-09-06T18:38:28.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.170+0000 7fc28a373780 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-09-06T18:38:28.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.170+0000 7fc28a373780 20 check_secure_mon_conn(): mode 1 is insecure 2023-09-06T18:38:28.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.174+0000 7fc26bae8700 20 reqs_thread_entry: start 2023-09-06T18:38:28.183 INFO:teuthology.orchestra.run.smithi099.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2023-09-06T18:38:28.184 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['LUKEWARM', 'STANDARD']}}] 2023-09-06T18:38:28.184 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2023-09-06T18:38:28.184 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2023-09-06T18:38:28.185 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2023-09-06T18:38:28.237 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setuser ceph since I am not root 2023-09-06T18:38:28.237 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setgroup ceph since I am not root 2023-09-06T18:38:28.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.245+0000 7fbddcdd7780 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-09-06T18:38:28.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.245+0000 7fbddcdd7780 20 check_secure_mon_conn(): mode 1 is insecure 2023-09-06T18:38:28.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.249+0000 7fbdbe54c700 20 reqs_thread_entry: start 2023-09-06T18:38:28.259 INFO:teuthology.orchestra.run.smithi099.stdout:{"id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"","notif_pool":"default.rgw.log:notif"} 2023-09-06T18:38:28.260 DEBUG:tasks.util.rgw: json result: {'id': 'd19ec83f-a74c-4fd7-8a2d-747094fce3d9', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2023-09-06T18:38:28.260 INFO:tasks.rgw:Configuring storage class type = FROZEN 2023-09-06T18:38:28.260 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2023-09-06T18:38:28.261 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2023-09-06T18:38:28.261 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2023-09-06T18:38:28.310 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setuser ceph since I am not root 2023-09-06T18:38:28.311 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setgroup ceph since I am not root 2023-09-06T18:38:28.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.319+0000 7f0a1079e780 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-09-06T18:38:28.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.319+0000 7f0a1079e780 20 check_secure_mon_conn(): mode 1 is insecure 2023-09-06T18:38:28.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.322+0000 7f09f1f13700 20 reqs_thread_entry: start 2023-09-06T18:38:28.331 INFO:teuthology.orchestra.run.smithi099.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2023-09-06T18:38:28.331 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['FROZEN', 'LUKEWARM', 'STANDARD']}}] 2023-09-06T18:38:28.331 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2023-09-06T18:38:28.332 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2023-09-06T18:38:28.332 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2023-09-06T18:38:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setuser ceph since I am not root 2023-09-06T18:38:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setgroup ceph since I am not root 2023-09-06T18:38:28.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.392+0000 7efc8026d780 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-09-06T18:38:28.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.392+0000 7efc8026d780 20 check_secure_mon_conn(): mode 1 is insecure 2023-09-06T18:38:28.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:38:28.396+0000 7efc619e2700 20 reqs_thread_entry: start 2023-09-06T18:38:28.406 INFO:teuthology.orchestra.run.smithi099.stdout:{"id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"","notif_pool":"default.rgw.log:notif"} 2023-09-06T18:38:28.407 DEBUG:tasks.util.rgw: json result: {'id': 'd19ec83f-a74c-4fd7-8a2d-747094fce3d9', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'FROZEN': {'data_pool': 'default.rgw.buckets.data.frozen'}, 'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2023-09-06T18:38:28.407 INFO:tasks.rgw:Starting rgw... 2023-09-06T18:38:28.407 INFO:tasks.rgw:rgw client.0 config is {} 2023-09-06T18:38:28.408 INFO:tasks.rgw:Using beast as radosgw frontend 2023-09-06T18:38:28.408 INFO:tasks.rgw.client.0:Restarting daemon 2023-09-06T18:38:28.409 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2023-09-06T18:38:28.411 INFO:tasks.rgw.client.0:Started 2023-09-06T18:38:28.412 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi099.front.sepia.ceph.com:80/ 2023-09-06T18:38:28.412 DEBUG:teuthology.orchestra.run.smithi099:> curl http://smithi099.front.sepia.ceph.com:80/ 2023-09-06T18:38:28.447 INFO:teuthology.orchestra.run.smithi099.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2023-09-06T18:38:28.448 INFO:teuthology.orchestra.run.smithi099.stderr: Dload Upload Total Spent Left Speed 2023-09-06T18:38:28.448 INFO:teuthology.orchestra.run.smithi099.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi099.front.sepia.ceph.com port 80: Connection refused 2023-09-06T18:38:28.449 DEBUG:teuthology.orchestra.run:got remote process result: 7 2023-09-06T18:38:29.450 DEBUG:teuthology.orchestra.run.smithi099:> curl http://smithi099.front.sepia.ceph.com:80/ 2023-09-06T18:38:29.473 INFO:teuthology.orchestra.run.smithi099.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2023-09-06T18:38:29.473 INFO:teuthology.orchestra.run.smithi099.stderr: Dload Upload Total Spent Left Speed 2023-09-06T18:38:29.474 INFO:teuthology.orchestra.run.smithi099.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 104k 0 --:--:-- --:--:-- --:--:-- 104k 2023-09-06T18:38:29.475 INFO:teuthology.orchestra.run.smithi099.stdout:anonymous 2023-09-06T18:38:29.476 INFO:teuthology.run_tasks:Running task rabbitmq... 2023-09-06T18:38:29.487 DEBUG:tasks.rabbitmq:RabbitMQ config is {'client.0': None} 2023-09-06T18:38:29.487 INFO:tasks.rabbitmq:Installing RabbitMQ... 2023-09-06T18:38:29.488 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install epel-release 2023-09-06T18:38:30.373 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:04:49 ago on Wed 06 Sep 2023 06:33:41 PM UTC. 2023-09-06T18:38:31.950 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:38:31.951 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:38:31.951 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2023-09-06T18:38:31.951 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:38:31.952 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:38:31.952 INFO:teuthology.orchestra.run.smithi099.stdout: epel-release noarch 8-19.el8 epel 25 k 2023-09-06T18:38:31.952 INFO:teuthology.orchestra.run.smithi099.stdout:Installing weak dependencies: 2023-09-06T18:38:31.952 INFO:teuthology.orchestra.run.smithi099.stdout: epel-next-release noarch 8-19.el8 epel 11 k 2023-09-06T18:38:31.952 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:31.953 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:38:31.953 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:38:31.953 INFO:teuthology.orchestra.run.smithi099.stdout:Install 2 Packages 2023-09-06T18:38:31.954 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:31.955 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 35 k 2023-09-06T18:38:31.955 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 37 k 2023-09-06T18:38:31.955 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:38:32.093 INFO:teuthology.orchestra.run.smithi099.stdout:(1/2): epel-next-release-8-19.el8.noarch.rpm 141 kB/s | 11 kB 00:00 2023-09-06T18:38:32.118 INFO:teuthology.orchestra.run.smithi099.stdout:(2/2): epel-release-8-19.el8.noarch.rpm 240 kB/s | 25 kB 00:00 2023-09-06T18:38:32.118 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:38:32.119 INFO:teuthology.orchestra.run.smithi099.stdout:Total 213 kB/s | 35 kB 00:00 2023-09-06T18:38:32.119 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:38:32.125 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:38:32.126 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:38:32.126 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:38:32.128 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:38:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:38:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : epel-release-8-19.el8.noarch 1/2 2023-09-06T18:38:32.210 INFO:teuthology.orchestra.run.smithi099.stdout:warning: /etc/yum.repos.d/epel-testing.repo created as /etc/yum.repos.d/epel-testing.repo.rpmnew 2023-09-06T18:38:32.211 INFO:teuthology.orchestra.run.smithi099.stdout:warning: /etc/yum.repos.d/epel.repo created as /etc/yum.repos.d/epel.repo.rpmnew 2023-09-06T18:38:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: epel-release-8-19.el8.noarch 1/2 2023-09-06T18:38:32.422 INFO:teuthology.orchestra.run.smithi099.stdout:Many EPEL packages require the CodeReady Builder (CRB) repository. 2023-09-06T18:38:32.422 INFO:teuthology.orchestra.run.smithi099.stdout:It is recommended that you run /usr/bin/crb enable to enable the CRB repository. 2023-09-06T18:38:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:32.584 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : epel-next-release-8-19.el8.noarch 2/2 2023-09-06T18:38:32.960 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: epel-next-release-8-19.el8.noarch 2/2 2023-09-06T18:38:32.960 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : epel-next-release-8-19.el8.noarch 1/2 2023-09-06T18:38:33.606 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : epel-release-8-19.el8.noarch 2/2 2023-09-06T18:38:33.607 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:33.607 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:38:33.607 INFO:teuthology.orchestra.run.smithi099.stdout: epel-next-release-8-19.el8.noarch epel-release-8-19.el8.noarch 2023-09-06T18:38:33.607 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:33.608 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:38:33.722 DEBUG:teuthology.orchestra.run.smithi099:> curl -s https://packagecloud.io/install/repositories/rabbitmq/erlang/script.rpm.sh | sudo bash 2023-09-06T18:38:34.111 INFO:teuthology.orchestra.run.smithi099.stdout:Detected operating system as centos/8. 2023-09-06T18:38:34.111 INFO:teuthology.orchestra.run.smithi099.stdout:Checking for curl... 2023-09-06T18:38:34.112 INFO:teuthology.orchestra.run.smithi099.stdout:Detected curl... 2023-09-06T18:38:34.112 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading repository file: https://packagecloud.io/install/repositories/rabbitmq/erlang/config_file.repo?os=centos&dist=8&source=script 2023-09-06T18:38:34.444 INFO:teuthology.orchestra.run.smithi099.stdout:done. 2023-09-06T18:38:34.445 INFO:teuthology.orchestra.run.smithi099.stdout:Installing pygpgme to verify GPG signatures... 2023-09-06T18:38:35.882 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 8 - Next - 616 kB/s | 368 kB 00:00 2023-09-06T18:38:37.502 INFO:teuthology.orchestra.run.smithi099.stdout:rabbitmq_erlang-source 608 B/s | 819 B 00:01 2023-09-06T18:38:37.881 INFO:teuthology.orchestra.run.smithi099.stdout:rabbitmq_erlang-source 10 kB/s | 3.8 kB 00:00 2023-09-06T18:38:37.940 INFO:teuthology.orchestra.run.smithi099.stderr:Importing GPG key 0xDF309A0B: 2023-09-06T18:38:37.940 INFO:teuthology.orchestra.run.smithi099.stderr: Userid : "https://packagecloud.io/rabbitmq/erlang (https://packagecloud.io/docs#gpg_signing) " 2023-09-06T18:38:37.941 INFO:teuthology.orchestra.run.smithi099.stderr: Fingerprint: 2EBD E413 D3CE 5D35 BCD1 5B7C 71C6 3471 DF30 9A0B 2023-09-06T18:38:37.941 INFO:teuthology.orchestra.run.smithi099.stderr: From : https://packagecloud.io/rabbitmq/erlang/gpgkey 2023-09-06T18:38:39.950 INFO:teuthology.orchestra.run.smithi099.stdout:rabbitmq_erlang-source 149 B/s | 296 B 00:01 2023-09-06T18:38:40.086 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:01 ago on Wed 06 Sep 2023 06:38:39 PM UTC. 2023-09-06T18:38:42.118 INFO:teuthology.orchestra.run.smithi099.stdout:No match for argument: pygpgme 2023-09-06T18:38:42.176 INFO:teuthology.orchestra.run.smithi099.stderr:Error: Unable to find a match: pygpgme 2023-09-06T18:38:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:WARNING: 2023-09-06T18:38:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:The pygpgme package could not be installed. This means GPG verification is not possible for any RPM installed on your system. 2023-09-06T18:38:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:To fix this, add a repository with pygpgme. Usualy, the EPEL repository for your system will have this. 2023-09-06T18:38:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:More information: https://fedoraproject.org/wiki/EPEL#How_can_I_use_these_extra_packages.3F 2023-09-06T18:38:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:43.372 INFO:teuthology.orchestra.run.smithi099.stdout:Installing yum-utils... 2023-09-06T18:38:44.653 INFO:teuthology.orchestra.run.smithi099.stdout:rabbitmq_erlang-source 2.1 kB/s | 951 B 00:00 2023-09-06T18:38:46.053 INFO:teuthology.orchestra.run.smithi099.stdout:Package yum-utils-4.0.21-23.el8.noarch is already installed. 2023-09-06T18:38:46.229 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:38:46.230 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2023-09-06T18:38:46.230 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:38:46.942 INFO:teuthology.orchestra.run.smithi099.stdout:Generating yum cache for rabbitmq_erlang... 2023-09-06T18:38:49.366 INFO:teuthology.orchestra.run.smithi099.stdout:Generating yum cache for rabbitmq_erlang-source... 2023-09-06T18:38:50.273 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:50.273 INFO:teuthology.orchestra.run.smithi099.stdout:The repository is setup! You can now install packages. 2023-09-06T18:38:50.275 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install erlang 2023-09-06T18:38:51.166 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:01 ago on Wed 06 Sep 2023 06:38:50 PM UTC. 2023-09-06T18:38:52.794 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:38:52.795 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:38:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2023-09-06T18:38:52.795 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:38:52.795 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:38:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: erlang x86_64 26.0.2-1.el8 rabbitmq_erlang 21 M 2023-09-06T18:38:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:52.796 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:38:52.796 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:38:52.796 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2023-09-06T18:38:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:52.797 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 21 M 2023-09-06T18:38:52.797 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 37 M 2023-09-06T18:38:52.797 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:38:53.607 INFO:teuthology.orchestra.run.smithi099.stdout:erlang-26.0.2-1.el8.x86_64.rpm 26 MB/s | 21 MB 00:00 2023-09-06T18:38:53.609 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:38:53.610 INFO:teuthology.orchestra.run.smithi099.stdout:Total 25 MB/s | 21 MB 00:00 2023-09-06T18:38:53.610 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:38:53.620 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:38:53.621 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:38:53.840 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:38:53.843 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:38:56.066 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:38:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : erlang-26.0.2-1.el8.x86_64 1/1 2023-09-06T18:38:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: erlang-26.0.2-1.el8.x86_64 1/1 2023-09-06T18:38:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : erlang-26.0.2-1.el8.x86_64 1/1 2023-09-06T18:38:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:57.312 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:38:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: erlang-26.0.2-1.el8.x86_64 2023-09-06T18:38:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:38:57.312 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:38:57.444 DEBUG:teuthology.orchestra.run.smithi099:> curl -s https://packagecloud.io/install/repositories/rabbitmq/rabbitmq-server/script.rpm.sh | sudo bash 2023-09-06T18:38:57.792 INFO:teuthology.orchestra.run.smithi099.stdout:Detected operating system as centos/8. 2023-09-06T18:38:57.792 INFO:teuthology.orchestra.run.smithi099.stdout:Checking for curl... 2023-09-06T18:38:57.792 INFO:teuthology.orchestra.run.smithi099.stdout:Detected curl... 2023-09-06T18:38:57.792 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading repository file: https://packagecloud.io/install/repositories/rabbitmq/rabbitmq-server/config_file.repo?os=centos&dist=8&source=script 2023-09-06T18:38:58.146 INFO:teuthology.orchestra.run.smithi099.stdout:done. 2023-09-06T18:38:58.147 INFO:teuthology.orchestra.run.smithi099.stdout:Installing pygpgme to verify GPG signatures... 2023-09-06T18:39:00.312 INFO:teuthology.orchestra.run.smithi099.stdout:rabbitmq_rabbitmq-server-source 620 B/s | 819 B 00:01 2023-09-06T18:39:00.776 INFO:teuthology.orchestra.run.smithi099.stdout:rabbitmq_rabbitmq-server-source 8.3 kB/s | 3.9 kB 00:00 2023-09-06T18:39:00.807 INFO:teuthology.orchestra.run.smithi099.stderr:Importing GPG key 0x4D206F89: 2023-09-06T18:39:00.807 INFO:teuthology.orchestra.run.smithi099.stderr: Userid : "https://packagecloud.io/rabbitmq/rabbitmq-server (https://packagecloud.io/docs#gpg_signing) " 2023-09-06T18:39:00.807 INFO:teuthology.orchestra.run.smithi099.stderr: Fingerprint: 8C69 5B02 19AF DEB0 4A05 8ED8 F4E7 8920 4D20 6F89 2023-09-06T18:39:00.808 INFO:teuthology.orchestra.run.smithi099.stderr: From : https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey 2023-09-06T18:39:02.778 INFO:teuthology.orchestra.run.smithi099.stdout:rabbitmq_rabbitmq-server-source 152 B/s | 296 B 00:01 2023-09-06T18:39:04.720 INFO:teuthology.orchestra.run.smithi099.stdout:No match for argument: pygpgme 2023-09-06T18:39:04.763 INFO:teuthology.orchestra.run.smithi099.stderr:Error: Unable to find a match: pygpgme 2023-09-06T18:39:05.986 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:39:05.986 INFO:teuthology.orchestra.run.smithi099.stdout:WARNING: 2023-09-06T18:39:05.987 INFO:teuthology.orchestra.run.smithi099.stdout:The pygpgme package could not be installed. This means GPG verification is not possible for any RPM installed on your system. 2023-09-06T18:39:05.987 INFO:teuthology.orchestra.run.smithi099.stdout:To fix this, add a repository with pygpgme. Usualy, the EPEL repository for your system will have this. 2023-09-06T18:39:05.987 INFO:teuthology.orchestra.run.smithi099.stdout:More information: https://fedoraproject.org/wiki/EPEL#How_can_I_use_these_extra_packages.3F 2023-09-06T18:39:05.987 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:39:05.988 INFO:teuthology.orchestra.run.smithi099.stdout:Installing yum-utils... 2023-09-06T18:39:07.518 INFO:teuthology.orchestra.run.smithi099.stdout:rabbitmq_rabbitmq-server-source 1.3 kB/s | 951 B 00:00 2023-09-06T18:39:08.899 INFO:teuthology.orchestra.run.smithi099.stdout:Package yum-utils-4.0.21-23.el8.noarch is already installed. 2023-09-06T18:39:09.081 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:39:09.082 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2023-09-06T18:39:09.082 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:39:09.787 INFO:teuthology.orchestra.run.smithi099.stdout:Generating yum cache for rabbitmq_rabbitmq-server... 2023-09-06T18:39:12.231 INFO:teuthology.orchestra.run.smithi099.stdout:Generating yum cache for rabbitmq_rabbitmq-server-source... 2023-09-06T18:39:13.125 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:39:13.125 INFO:teuthology.orchestra.run.smithi099.stdout:The repository is setup! You can now install packages. 2023-09-06T18:39:13.126 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install rabbitmq-server 2023-09-06T18:39:14.010 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:01 ago on Wed 06 Sep 2023 06:39:13 PM UTC. 2023-09-06T18:39:15.730 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:39:15.731 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:39:15.732 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-06T18:39:15.732 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:39:15.732 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:39:15.732 INFO:teuthology.orchestra.run.smithi099.stdout: rabbitmq-server noarch 3.12.4-1.el8 rabbitmq_rabbitmq-server 17 M 2023-09-06T18:39:15.732 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:39:15.733 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:39:15.733 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:39:15.733 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2023-09-06T18:39:15.733 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:39:15.733 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 17 M 2023-09-06T18:39:15.734 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 22 M 2023-09-06T18:39:15.734 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:39:16.526 INFO:teuthology.orchestra.run.smithi099.stdout:rabbitmq-server-3.12.4-1.el8.noarch.rpm 21 MB/s | 17 MB 00:00 2023-09-06T18:39:16.528 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:39:16.528 INFO:teuthology.orchestra.run.smithi099.stdout:Total 21 MB/s | 17 MB 00:00 2023-09-06T18:39:16.529 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:39:16.540 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:39:16.541 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:39:16.633 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:39:16.636 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:39:16.798 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:39:18.052 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rabbitmq-server-3.12.4-1.el8.noarch 1/1 2023-09-06T18:39:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rabbitmq-server-3.12.4-1.el8.noarch 1/1 2023-09-06T18:39:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rabbitmq-server-3.12.4-1.el8.noarch 1/1 2023-09-06T18:39:18.758 INFO:teuthology.orchestra.run.smithi099.stdout:[/usr/lib/tmpfiles.d/rabbitmq-server.conf:1] Line references path below legacy directory /var/run/, updating /var/run/rabbitmq → /run/rabbitmq; please update the tmpfiles.d/ drop-in file accordingly. 2023-09-06T18:39:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:39:19.638 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rabbitmq-server-3.12.4-1.el8.noarch 1/1 2023-09-06T18:39:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:39:19.639 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:39:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: rabbitmq-server-3.12.4-1.el8.noarch 2023-09-06T18:39:19.640 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:39:19.640 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:39:19.866 INFO:tasks.rabbitmq:Bringing up Daemon and RabbitMQ service... 2023-09-06T18:39:19.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo chkconfig rabbitmq-server on 2023-09-06T18:39:19.903 INFO:teuthology.orchestra.run.smithi099.stderr:Note: Forwarding request to 'systemctl enable rabbitmq-server.service'. 2023-09-06T18:39:19.908 INFO:teuthology.orchestra.run.smithi099.stderr:Created symlink /etc/systemd/system/multi-user.target.wants/rabbitmq-server.service → /usr/lib/systemd/system/rabbitmq-server.service. 2023-09-06T18:39:20.009 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/service rabbitmq-server start 2023-09-06T18:39:20.087 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl start rabbitmq-server.service 2023-09-06T18:39:23.881 INFO:teuthology.run_tasks:Running task notification-tests... 2023-09-06T18:39:23.894 DEBUG:tasks.notification_tests:Notifications config is {'client.0': {'extra_attr': ['amqp_test'], 'rgw_server': 'client.0'}} 2023-09-06T18:39:23.895 INFO:tasks.notification_tests:Downloading bucket-notifications-tests... 2023-09-06T18:39:23.895 INFO:tasks.notification_tests:Using branch wip-eric-testing-1 from https://git.ceph.com/ceph-ci.git for bucket notifications tests 2023-09-06T18:39:23.896 DEBUG:teuthology.orchestra.run.smithi099:> git clone -b wip-eric-testing-1 https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/ceph 2023-09-06T18:39:23.940 INFO:teuthology.orchestra.run.smithi099.stderr:Cloning into '/home/ubuntu/cephtest/ceph'... 2023-09-06T18:40:20.870 INFO:tasks.notification_tests:Pre-processing... 2023-09-06T18:40:20.871 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p /home/ubuntu/.aws/models/s3/2006-03-01/ 2023-09-06T18:40:20.934 DEBUG:teuthology.orchestra.run.smithi099:> cd /home/ubuntu/.aws/models/s3/2006-03-01/ && cp /home/ubuntu/cephtest/ceph/examples/rgw/boto3/service-2.sdk-extras.json service-2.sdk-extras.json 2023-09-06T18:40:20.992 INFO:tasks.notification_tests:Creating rgw user... 2023-09-06T18:40:20.993 DEBUG:tasks.notification_tests:Creating user foo.client.0 on client.0 2023-09-06T18:40:20.993 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name 'Mr. foo.client.0' --access-key DPCRSOKICJWGAVMTJRSS --secret N8aicnhqv2R9aAh1uOWvAzP5S/sy2uws/dSYbjVJLexlHpacTavu0g== --cluster ceph 2023-09-06T18:40:21.083 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setuser ceph since I am not root 2023-09-06T18:40:21.084 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setgroup ceph since I am not root 2023-09-06T18:40:21.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.085+0000 7fc21b4bf780 -1 asok(0x5639f36e4000) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-client.0.125467.asok': (13) Permission denied 2023-09-06T18:40:21.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.092+0000 7fc21b4bf780 20 rados->read ofs=0 len=0 2023-09-06T18:40:21.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.093+0000 7fc21b4bf780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:40:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.093+0000 7fc21b4bf780 20 realm 2023-09-06T18:40:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.093+0000 7fc21b4bf780 20 rados->read ofs=0 len=0 2023-09-06T18:40:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.093+0000 7fc21b4bf780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:40:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.093+0000 7fc21b4bf780 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-09-06T18:40:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.093+0000 7fc21b4bf780 20 rados->read ofs=0 len=0 2023-09-06T18:40:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.093+0000 7fc21b4bf780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:40:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.093+0000 7fc21b4bf780 20 rados->read ofs=0 len=0 2023-09-06T18:40:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.094+0000 7fc21b4bf780 20 rados_obj.operate() r=0 bl.length=46 2023-09-06T18:40:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.094+0000 7fc21b4bf780 20 rados->read ofs=0 len=0 2023-09-06T18:40:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.095+0000 7fc21b4bf780 20 rados_obj.operate() r=0 bl.length=1004 2023-09-06T18:40:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.095+0000 7fc21b4bf780 20 searching for the correct realm 2023-09-06T18:40:21.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.107+0000 7fc21b4bf780 20 RGWRados::pool_iterate: got zonegroup_info.aafaa245-59dc-4ab8-adab-a5293aa3a8a6 2023-09-06T18:40:21.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.107+0000 7fc21b4bf780 20 RGWRados::pool_iterate: got zone_names.default 2023-09-06T18:40:21.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.107+0000 7fc21b4bf780 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-09-06T18:40:21.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.107+0000 7fc21b4bf780 20 RGWRados::pool_iterate: got zone_info.d19ec83f-a74c-4fd7-8a2d-747094fce3d9 2023-09-06T18:40:21.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.107+0000 7fc21b4bf780 20 rados->read ofs=0 len=0 2023-09-06T18:40:21.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.107+0000 7fc21b4bf780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:40:21.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.107+0000 7fc21b4bf780 20 rados->read ofs=0 len=0 2023-09-06T18:40:21.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.108+0000 7fc21b4bf780 20 rados_obj.operate() r=0 bl.length=46 2023-09-06T18:40:21.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.108+0000 7fc21b4bf780 20 rados->read ofs=0 len=0 2023-09-06T18:40:21.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.108+0000 7fc21b4bf780 20 rados_obj.operate() r=0 bl.length=454 2023-09-06T18:40:21.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.108+0000 7fc21b4bf780 20 zone default found 2023-09-06T18:40:21.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.108+0000 7fc21b4bf780 4 Realm: () 2023-09-06T18:40:21.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.108+0000 7fc21b4bf780 4 ZoneGroup: default (aafaa245-59dc-4ab8-adab-a5293aa3a8a6) 2023-09-06T18:40:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.108+0000 7fc21b4bf780 4 Zone: default (d19ec83f-a74c-4fd7-8a2d-747094fce3d9) 2023-09-06T18:40:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.108+0000 7fc21b4bf780 10 cannot find current period zonegroup using local zonegroup configuration 2023-09-06T18:40:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.108+0000 7fc21b4bf780 20 zonegroup default 2023-09-06T18:40:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.108+0000 7fc21b4bf780 20 rados->read ofs=0 len=0 2023-09-06T18:40:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.108+0000 7fc21b4bf780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:40:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.108+0000 7fc21b4bf780 20 rados->read ofs=0 len=0 2023-09-06T18:40:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.109+0000 7fc21b4bf780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:40:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.109+0000 7fc21b4bf780 20 rados->read ofs=0 len=0 2023-09-06T18:40:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.109+0000 7fc21b4bf780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:40:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.109+0000 7fc21b4bf780 20 started sync module instance, tier type = 2023-09-06T18:40:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.109+0000 7fc21b4bf780 20 started zone id=d19ec83f-a74c-4fd7-8a2d-747094fce3d9 (name=default) with tier type = 2023-09-06T18:40:21.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.122+0000 7fc21b4bf780 20 add_watcher() i=0 2023-09-06T18:40:21.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.122+0000 7fc21b4bf780 20 add_watcher() i=1 2023-09-06T18:40:21.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.122+0000 7fc21b4bf780 20 add_watcher() i=2 2023-09-06T18:40:21.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.122+0000 7fc21b4bf780 20 add_watcher() i=3 2023-09-06T18:40:21.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.122+0000 7fc21b4bf780 20 add_watcher() i=4 2023-09-06T18:40:21.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.122+0000 7fc21b4bf780 20 add_watcher() i=5 2023-09-06T18:40:21.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.122+0000 7fc21b4bf780 20 add_watcher() i=6 2023-09-06T18:40:21.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.125+0000 7fc21b4bf780 20 add_watcher() i=7 2023-09-06T18:40:21.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.125+0000 7fc21b4bf780 2 all 8 watchers are set, enabling cache 2023-09-06T18:40:21.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.126+0000 7fc200c3c700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-09-06T18:40:21.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.126+0000 7fc21b4bf780 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-09-06T18:40:21.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.126+0000 7fc21b4bf780 20 check_secure_mon_conn(): mode 1 is insecure 2023-09-06T18:40:21.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.131+0000 7fc21b4bf780 5 note: GC not initialized 2023-09-06T18:40:21.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.132+0000 7fc1fc433700 20 reqs_thread_entry: start 2023-09-06T18:40:21.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.132+0000 7fc21b4bf780 20 init_complete bucket index max shards: 11 2023-09-06T18:40:21.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.132+0000 7fc1fa42f700 20 reqs_thread_entry: start 2023-09-06T18:40:21.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.132+0000 7fc21b4bf780 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2023-09-06T18:40:21.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.132+0000 7fc21b4bf780 20 rados->read ofs=0 len=0 2023-09-06T18:40:21.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.132+0000 7fc21b4bf780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:40:21.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.132+0000 7fc21b4bf780 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2023-09-06T18:40:21.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.132+0000 7fc21b4bf780 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2023-09-06T18:40:21.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.132+0000 7fc21b4bf780 10 cache get: name=default.rgw.meta+users.keys+DPCRSOKICJWGAVMTJRSS : miss 2023-09-06T18:40:21.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.132+0000 7fc21b4bf780 20 rados->read ofs=0 len=0 2023-09-06T18:40:21.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.133+0000 7fc21b4bf780 20 rados_obj.operate() r=-2 bl.length=0 2023-09-06T18:40:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.133+0000 7fc21b4bf780 10 cache put: name=default.rgw.meta+users.keys+DPCRSOKICJWGAVMTJRSS info.flags=0x0 2023-09-06T18:40:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.133+0000 7fc21b4bf780 10 adding default.rgw.meta+users.keys+DPCRSOKICJWGAVMTJRSS to cache LRU end 2023-09-06T18:40:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.133+0000 7fc21b4bf780 10 cache get: name=default.rgw.meta+users.keys+DPCRSOKICJWGAVMTJRSS : hit (negative entry) 2023-09-06T18:40:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.133+0000 7fc21b4bf780 10 cache get: name=default.rgw.meta+users.keys+DPCRSOKICJWGAVMTJRSS : hit (negative entry) 2023-09-06T18:40:21.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.134+0000 7fc21b4bf780 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2023-09-06T18:40:21.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.134+0000 7fc21b4bf780 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2023-09-06T18:40:21.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.134+0000 7fc21b4bf780 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2023-09-06T18:40:21.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.134+0000 7fc214463700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 146028888064 cookie 94806899452928 notifier 4551 bl.length()=594 2023-09-06T18:40:21.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.134+0000 7fc214463700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2023-09-06T18:40:21.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.134+0000 7fc214463700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2023-09-06T18:40:21.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.136+0000 7fc21b4bf780 10 cache put: name=default.rgw.meta+users.keys+DPCRSOKICJWGAVMTJRSS info.flags=0x7 2023-09-06T18:40:21.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.136+0000 7fc21b4bf780 10 moving default.rgw.meta+users.keys+DPCRSOKICJWGAVMTJRSS to cache LRU end 2023-09-06T18:40:21.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.136+0000 7fc21b4bf780 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:DPCRSOKICJWGAVMTJRSS, ofs0, ns] 2023-09-06T18:40:21.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.136+0000 7fc214463700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 146028888064 cookie 94806899454976 notifier 4551 bl.length()=186 2023-09-06T18:40:21.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.136+0000 7fc214463700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+DPCRSOKICJWGAVMTJRSS info.flags=0x7 2023-09-06T18:40:21.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.136+0000 7fc214463700 10 rgw watcher librados: moving default.rgw.meta+users.keys+DPCRSOKICJWGAVMTJRSS to cache LRU end 2023-09-06T18:40:21.141 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-09-06T18:40:21.141 INFO:teuthology.orchestra.run.smithi099.stdout: "user_id": "foo.client.0", 2023-09-06T18:40:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: "display_name": "Mr. foo.client.0", 2023-09-06T18:40:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: "email": "", 2023-09-06T18:40:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: "suspended": 0, 2023-09-06T18:40:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: "max_buckets": 1000, 2023-09-06T18:40:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: "subusers": [], 2023-09-06T18:40:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: "keys": [ 2023-09-06T18:40:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-09-06T18:40:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: "user": "foo.client.0", 2023-09-06T18:40:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: "access_key": "DPCRSOKICJWGAVMTJRSS", 2023-09-06T18:40:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: "secret_key": "N8aicnhqv2R9aAh1uOWvAzP5S/sy2uws/dSYbjVJLexlHpacTavu0g==" 2023-09-06T18:40:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-06T18:40:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-09-06T18:40:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: "swift_keys": [], 2023-09-06T18:40:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": [], 2023-09-06T18:40:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: "op_mask": "read, write, delete", 2023-09-06T18:40:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "default_placement": "", 2023-09-06T18:40:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "default_storage_class": "", 2023-09-06T18:40:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "placement_tags": [], 2023-09-06T18:40:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "bucket_quota": { 2023-09-06T18:40:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": false, 2023-09-06T18:40:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: "check_on_raw": false, 2023-09-06T18:40:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: "max_size": -1, 2023-09-06T18:40:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: "max_size_kb": 0, 2023-09-06T18:40:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: "max_objects": -1 2023-09-06T18:40:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-06T18:40:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: "user_quota": { 2023-09-06T18:40:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": false, 2023-09-06T18:40:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: "check_on_raw": false, 2023-09-06T18:40:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: "max_size": -1, 2023-09-06T18:40:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: "max_size_kb": 0, 2023-09-06T18:40:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: "max_objects": -1 2023-09-06T18:40:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-06T18:40:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: "temp_url_keys": [], 2023-09-06T18:40:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "rgw", 2023-09-06T18:40:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: "mfa_ids": [] 2023-09-06T18:40:21.149 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-09-06T18:40:21.149 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:40:21.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.137+0000 7fc200c3c700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-09-06T18:40:21.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.139+0000 7fc21b4bf780 20 remove_watcher() i=0 2023-09-06T18:40:21.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.139+0000 7fc21b4bf780 2 removed watcher, disabling cache 2023-09-06T18:40:21.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.140+0000 7fc21b4bf780 20 remove_watcher() i=1 2023-09-06T18:40:21.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.141+0000 7fc21b4bf780 20 remove_watcher() i=2 2023-09-06T18:40:21.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.143+0000 7fc21b4bf780 20 remove_watcher() i=3 2023-09-06T18:40:21.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.144+0000 7fc21b4bf780 20 remove_watcher() i=4 2023-09-06T18:40:21.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.145+0000 7fc21b4bf780 20 remove_watcher() i=5 2023-09-06T18:40:21.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.147+0000 7fc21b4bf780 20 remove_watcher() i=6 2023-09-06T18:40:21.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T18:40:21.148+0000 7fc21b4bf780 20 remove_watcher() i=7 2023-09-06T18:40:21.154 INFO:tasks.notification_tests:Configuring bucket-notifications-tests... 2023-09-06T18:40:21.154 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-06T18:40:21.155 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/bn-tests.client.0.conf 2023-09-06T18:40:21.173 DEBUG:teuthology.orchestra.run.smithi099:> cd /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification && ./bootstrap 2023-09-06T18:40:30.348 INFO:teuthology.orchestra.run.smithi099.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2023-09-06T18:40:31.203 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:18 ago on Wed 06 Sep 2023 06:39:13 PM UTC. 2023-09-06T18:40:32.572 INFO:teuthology.orchestra.run.smithi099.stdout:Package python36-devel-3.6.8-38.module_el8.5.0+895+a459eca8.x86_64 is already installed. 2023-09-06T18:40:32.770 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:40:32.773 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:40:32.773 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-06T18:40:32.773 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:40:32.774 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-06T18:40:32.774 INFO:teuthology.orchestra.run.smithi099.stdout: libevent-devel x86_64 2.1.8-5.el8 appstream 104 k 2023-09-06T18:40:32.774 INFO:teuthology.orchestra.run.smithi099.stdout: libxml2-devel x86_64 2.9.7-16.el8 appstream 1.0 M 2023-09-06T18:40:32.774 INFO:teuthology.orchestra.run.smithi099.stdout: libxslt-devel x86_64 1.1.32-6.el8 appstream 323 k 2023-09-06T18:40:32.774 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pika noarch 1.0.1-3.el8.1 epel 258 k 2023-09-06T18:40:32.775 INFO:teuthology.orchestra.run.smithi099.stdout: python3-xmltodict noarch 0.12.0-6.el8 epel 24 k 2023-09-06T18:40:32.775 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-25.el8 baseos 59 k 2023-09-06T18:40:32.775 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2023-09-06T18:40:32.775 INFO:teuthology.orchestra.run.smithi099.stdout: zlib x86_64 1.2.11-25.el8 baseos 103 k 2023-09-06T18:40:32.775 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-09-06T18:40:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: cmake-filesystem x86_64 3.20.2-5.el8 appstream 45 k 2023-09-06T18:40:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: libgcrypt-devel x86_64 1.8.5-7.el8 baseos 150 k 2023-09-06T18:40:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: libgpg-error-devel x86_64 1.31-1.el8 baseos 64 k 2023-09-06T18:40:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: python3-Automat noarch 0.7.0-3.1.el8 epel 66 k 2023-09-06T18:40:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs noarch 17.4.0-6.el8 appstream 53 k 2023-09-06T18:40:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: python3-constantly noarch 15.1.0-4.el8 epel 19 k 2023-09-06T18:40:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: python3-hamcrest noarch 1.9.0-8.el8 epel 75 k 2023-09-06T18:40:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: python3-hyperlink noarch 19.0.0-1.el8.1 epel 75 k 2023-09-06T18:40:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: python3-incremental noarch 17.5.0-5.el8.1 epel 33 k 2023-09-06T18:40:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycurl x86_64 7.43.0.2-4.el8 appstream 227 k 2023-09-06T18:40:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zope-event noarch 4.2.0-12.el8 epel 210 k 2023-09-06T18:40:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zope-interface x86_64 4.6.0-1.el8 epel 158 k 2023-09-06T18:40:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: xz-devel x86_64 5.2.4-4.el8 baseos 63 k 2023-09-06T18:40:32.778 INFO:teuthology.orchestra.run.smithi099.stdout:Installing weak dependencies: 2023-09-06T18:40:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: python3-service-identity noarch 18.1.0-2.el8 epel 28 k 2023-09-06T18:40:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tornado x86_64 6.0.2-1.el8 epel 722 k 2023-09-06T18:40:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: python3-twisted x86_64 19.10.0-4.el8 epel 6.8 M 2023-09-06T18:40:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:40:32.780 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:40:32.780 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:40:32.780 INFO:teuthology.orchestra.run.smithi099.stdout:Install 22 Packages 2023-09-06T18:40:32.780 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 1 Package 2023-09-06T18:40:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:40:32.781 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 11 M 2023-09-06T18:40:32.781 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-06T18:40:33.569 INFO:teuthology.orchestra.run.smithi099.stdout:(1/23): cmake-filesystem-3.20.2-5.el8.x86_64.rp 85 kB/s | 45 kB 00:00 2023-09-06T18:40:33.902 INFO:teuthology.orchestra.run.smithi099.stdout:(2/23): libevent-devel-2.1.8-5.el8.x86_64.rpm 121 kB/s | 104 kB 00:00 2023-09-06T18:40:34.620 INFO:teuthology.orchestra.run.smithi099.stdout:(3/23): python3-attrs-17.4.0-6.el8.noarch.rpm 74 kB/s | 53 kB 00:00 2023-09-06T18:40:34.670 INFO:teuthology.orchestra.run.smithi099.stdout:(4/23): libxml2-devel-2.9.7-16.el8.x86_64.rpm 655 kB/s | 1.0 MB 00:01 2023-09-06T18:40:34.945 INFO:teuthology.orchestra.run.smithi099.stdout:(5/23): libgcrypt-devel-1.8.5-7.el8.x86_64.rpm 544 kB/s | 150 kB 00:00 2023-09-06T18:40:34.971 INFO:teuthology.orchestra.run.smithi099.stdout:(6/23): libxslt-devel-1.1.32-6.el8.x86_64.rpm 230 kB/s | 323 kB 00:01 2023-09-06T18:40:35.163 INFO:teuthology.orchestra.run.smithi099.stdout:(7/23): libgpg-error-devel-1.31-1.el8.x86_64.rp 294 kB/s | 64 kB 00:00 2023-09-06T18:40:35.606 INFO:teuthology.orchestra.run.smithi099.stdout:(8/23): python3-pycurl-7.43.0.2-4.el8.x86_64.rp 231 kB/s | 227 kB 00:00 2023-09-06T18:40:35.730 INFO:teuthology.orchestra.run.smithi099.stdout:(9/23): zlib-devel-1.2.11-25.el8.x86_64.rpm 103 kB/s | 59 kB 00:00 2023-09-06T18:40:35.755 INFO:teuthology.orchestra.run.smithi099.stdout:(10/23): xz-devel-5.2.4-4.el8.x86_64.rpm 80 kB/s | 63 kB 00:00 2023-09-06T18:40:35.956 INFO:teuthology.orchestra.run.smithi099.stdout:(11/23): python3-Automat-0.7.0-3.1.el8.noarch.r 189 kB/s | 66 kB 00:00 2023-09-06T18:40:35.981 INFO:teuthology.orchestra.run.smithi099.stdout:(12/23): python3-constantly-15.1.0-4.el8.noarch 77 kB/s | 19 kB 00:00 2023-09-06T18:40:36.064 INFO:teuthology.orchestra.run.smithi099.stdout:(13/23): python3-hyperlink-19.0.0-1.el8.1.noarc 694 kB/s | 75 kB 00:00 2023-09-06T18:40:36.090 INFO:teuthology.orchestra.run.smithi099.stdout:(14/23): python3-hamcrest-1.9.0-8.el8.noarch.rp 224 kB/s | 75 kB 00:00 2023-09-06T18:40:36.106 INFO:teuthology.orchestra.run.smithi099.stdout:(15/23): python3-incremental-17.5.0-5.el8.1.noa 260 kB/s | 33 kB 00:00 2023-09-06T18:40:36.198 INFO:teuthology.orchestra.run.smithi099.stdout:(16/23): python3-service-identity-18.1.0-2.el8. 308 kB/s | 28 kB 00:00 2023-09-06T18:40:36.241 INFO:teuthology.orchestra.run.smithi099.stdout:(17/23): python3-pika-1.0.1-3.el8.1.noarch.rpm 1.4 MB/s | 258 kB 00:00 2023-09-06T18:40:36.374 INFO:teuthology.orchestra.run.smithi099.stdout:(18/23): python3-xmltodict-0.12.0-6.el8.noarch. 183 kB/s | 24 kB 00:00 2023-09-06T18:40:36.516 INFO:teuthology.orchestra.run.smithi099.stdout:(19/23): python3-zope-event-4.2.0-12.el8.noarch 1.5 MB/s | 210 kB 00:00 2023-09-06T18:40:36.541 INFO:teuthology.orchestra.run.smithi099.stdout:(20/23): python3-tornado-6.0.2-1.el8.x86_64.rpm 1.6 MB/s | 722 kB 00:00 2023-09-06T18:40:36.650 INFO:teuthology.orchestra.run.smithi099.stdout:(21/23): python3-zope-interface-4.6.0-1.el8.x86 1.2 MB/s | 158 kB 00:00 2023-09-06T18:40:36.675 INFO:teuthology.orchestra.run.smithi099.stdout:(22/23): zlib-1.2.11-25.el8.x86_64.rpm 774 kB/s | 103 kB 00:00 2023-09-06T18:40:36.851 INFO:teuthology.orchestra.run.smithi099.stdout:(23/23): python3-twisted-19.10.0-4.el8.x86_64.r 10 MB/s | 6.8 MB 00:00 2023-09-06T18:40:36.851 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-06T18:40:36.851 INFO:teuthology.orchestra.run.smithi099.stdout:Total 2.6 MB/s | 11 MB 00:04 2023-09-06T18:40:36.954 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:40:37.098 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:40:37.098 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:40:37.342 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:40:37.345 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:40:38.284 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:40:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-attrs-17.4.0-6.el8.noarch 1/24 2023-09-06T18:40:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgpg-error-devel-1.31-1.el8.x86_64 2/24 2023-09-06T18:40:38.776 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libgpg-error-devel-1.31-1.el8.x86_64 2/24 2023-09-06T18:40:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgcrypt-devel-1.8.5-7.el8.x86_64 3/24 2023-09-06T18:40:39.035 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libgcrypt-devel-1.8.5-7.el8.x86_64 3/24 2023-09-06T18:40:39.293 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-Automat-0.7.0-3.1.el8.noarch 4/24 2023-09-06T18:40:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-service-identity-18.1.0-2.el8.noarch 5/24 2023-09-06T18:40:39.804 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : zlib-1.2.11-25.el8.x86_64 6/24 2023-09-06T18:40:40.030 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-25.el8.x86_64 7/24 2023-09-06T18:40:40.329 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zope-event-4.2.0-12.el8.noarch 8/24 2023-09-06T18:40:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zope-interface-4.6.0-1.el8.x86_64 9/24 2023-09-06T18:40:40.929 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-incremental-17.5.0-5.el8.1.noarch 10/24 2023-09-06T18:40:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-hyperlink-19.0.0-1.el8.1.noarch 11/24 2023-09-06T18:40:41.606 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-hamcrest-1.9.0-8.el8.noarch 12/24 2023-09-06T18:40:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-constantly-15.1.0-4.el8.noarch 13/24 2023-09-06T18:40:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-twisted-19.10.0-4.el8.x86_64 14/24 2023-09-06T18:40:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xz-devel-5.2.4-4.el8.x86_64 15/24 2023-09-06T18:40:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycurl-7.43.0.2-4.el8.x86_64 16/24 2023-09-06T18:40:44.970 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tornado-6.0.2-1.el8.x86_64 17/24 2023-09-06T18:40:45.435 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cmake-filesystem-3.20.2-5.el8.x86_64 18/24 2023-09-06T18:40:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libxml2-devel-2.9.7-16.el8.x86_64 19/24 2023-09-06T18:40:46.088 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libxslt-devel-1.1.32-6.el8.x86_64 20/24 2023-09-06T18:40:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pika-1.0.1-3.el8.1.noarch 21/24 2023-09-06T18:40:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-xmltodict-0.12.0-6.el8.noarch 22/24 2023-09-06T18:40:46.914 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libevent-devel-2.1.8-5.el8.x86_64 23/24 2023-09-06T18:40:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : zlib-1.2.11-21.el8.x86_64 24/24 2023-09-06T18:40:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: zlib-1.2.11-21.el8.x86_64 24/24 2023-09-06T18:40:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cmake-filesystem-3.20.2-5.el8.x86_64 1/24 2023-09-06T18:40:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libevent-devel-2.1.8-5.el8.x86_64 2/24 2023-09-06T18:40:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libxml2-devel-2.9.7-16.el8.x86_64 3/24 2023-09-06T18:40:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libxslt-devel-1.1.32-6.el8.x86_64 4/24 2023-09-06T18:40:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-attrs-17.4.0-6.el8.noarch 5/24 2023-09-06T18:40:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycurl-7.43.0.2-4.el8.x86_64 6/24 2023-09-06T18:40:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgcrypt-devel-1.8.5-7.el8.x86_64 7/24 2023-09-06T18:40:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgpg-error-devel-1.31-1.el8.x86_64 8/24 2023-09-06T18:40:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xz-devel-5.2.4-4.el8.x86_64 9/24 2023-09-06T18:40:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-25.el8.x86_64 10/24 2023-09-06T18:40:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-Automat-0.7.0-3.1.el8.noarch 11/24 2023-09-06T18:40:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-constantly-15.1.0-4.el8.noarch 12/24 2023-09-06T18:40:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-hamcrest-1.9.0-8.el8.noarch 13/24 2023-09-06T18:40:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-hyperlink-19.0.0-1.el8.1.noarch 14/24 2023-09-06T18:40:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-incremental-17.5.0-5.el8.1.noarch 15/24 2023-09-06T18:40:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pika-1.0.1-3.el8.1.noarch 16/24 2023-09-06T18:40:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-service-identity-18.1.0-2.el8.noarch 17/24 2023-09-06T18:40:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tornado-6.0.2-1.el8.x86_64 18/24 2023-09-06T18:40:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-twisted-19.10.0-4.el8.x86_64 19/24 2023-09-06T18:40:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-xmltodict-0.12.0-6.el8.noarch 20/24 2023-09-06T18:40:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zope-event-4.2.0-12.el8.noarch 21/24 2023-09-06T18:40:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zope-interface-4.6.0-1.el8.x86_64 22/24 2023-09-06T18:40:47.484 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-1.2.11-25.el8.x86_64 23/24 2023-09-06T18:40:48.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-1.2.11-21.el8.x86_64 24/24 2023-09-06T18:40:48.383 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:40:48.383 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2023-09-06T18:40:48.384 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-1.2.11-25.el8.x86_64 2023-09-06T18:40:48.384 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-06T18:40:48.384 INFO:teuthology.orchestra.run.smithi099.stdout: cmake-filesystem-3.20.2-5.el8.x86_64 2023-09-06T18:40:48.384 INFO:teuthology.orchestra.run.smithi099.stdout: libevent-devel-2.1.8-5.el8.x86_64 2023-09-06T18:40:48.385 INFO:teuthology.orchestra.run.smithi099.stdout: libgcrypt-devel-1.8.5-7.el8.x86_64 2023-09-06T18:40:48.385 INFO:teuthology.orchestra.run.smithi099.stdout: libgpg-error-devel-1.31-1.el8.x86_64 2023-09-06T18:40:48.385 INFO:teuthology.orchestra.run.smithi099.stdout: libxml2-devel-2.9.7-16.el8.x86_64 2023-09-06T18:40:48.386 INFO:teuthology.orchestra.run.smithi099.stdout: libxslt-devel-1.1.32-6.el8.x86_64 2023-09-06T18:40:48.386 INFO:teuthology.orchestra.run.smithi099.stdout: python3-Automat-0.7.0-3.1.el8.noarch 2023-09-06T18:40:48.386 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs-17.4.0-6.el8.noarch 2023-09-06T18:40:48.386 INFO:teuthology.orchestra.run.smithi099.stdout: python3-constantly-15.1.0-4.el8.noarch 2023-09-06T18:40:48.387 INFO:teuthology.orchestra.run.smithi099.stdout: python3-hamcrest-1.9.0-8.el8.noarch 2023-09-06T18:40:48.387 INFO:teuthology.orchestra.run.smithi099.stdout: python3-hyperlink-19.0.0-1.el8.1.noarch 2023-09-06T18:40:48.387 INFO:teuthology.orchestra.run.smithi099.stdout: python3-incremental-17.5.0-5.el8.1.noarch 2023-09-06T18:40:48.388 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pika-1.0.1-3.el8.1.noarch 2023-09-06T18:40:48.388 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycurl-7.43.0.2-4.el8.x86_64 2023-09-06T18:40:48.389 INFO:teuthology.orchestra.run.smithi099.stdout: python3-service-identity-18.1.0-2.el8.noarch 2023-09-06T18:40:48.389 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tornado-6.0.2-1.el8.x86_64 2023-09-06T18:40:48.389 INFO:teuthology.orchestra.run.smithi099.stdout: python3-twisted-19.10.0-4.el8.x86_64 2023-09-06T18:40:48.390 INFO:teuthology.orchestra.run.smithi099.stdout: python3-xmltodict-0.12.0-6.el8.noarch 2023-09-06T18:40:48.390 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zope-event-4.2.0-12.el8.noarch 2023-09-06T18:40:48.391 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zope-interface-4.6.0-1.el8.x86_64 2023-09-06T18:40:48.391 INFO:teuthology.orchestra.run.smithi099.stdout: xz-devel-5.2.4-4.el8.x86_64 2023-09-06T18:40:48.391 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-25.el8.x86_64 2023-09-06T18:40:48.392 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:40:48.392 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:40:50.776 INFO:teuthology.orchestra.run.smithi099.stdout:Collecting pip 2023-09-06T18:40:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: Downloading https://files.pythonhosted.org/packages/a4/6d/6463d49a933f547439d6b5b98b46af8742cc03ae83543e4d7688c2420f8b/pip-21.3.1-py3-none-any.whl (1.7MB) 2023-09-06T18:40:51.151 INFO:teuthology.orchestra.run.smithi099.stdout:Installing collected packages: pip 2023-09-06T18:40:51.151 INFO:teuthology.orchestra.run.smithi099.stdout: Found existing installation: pip 9.0.3 2023-09-06T18:40:51.307 INFO:teuthology.orchestra.run.smithi099.stdout: Uninstalling pip-9.0.3: 2023-09-06T18:40:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: Successfully uninstalled pip-9.0.3 2023-09-06T18:40:51.982 INFO:teuthology.orchestra.run.smithi099.stdout:Successfully installed pip-21.3.1 2023-09-06T18:40:52.229 INFO:teuthology.orchestra.run.smithi099.stderr:You are using pip version 21.3.1, however version 23.2.1 is available. 2023-09-06T18:40:52.230 INFO:teuthology.orchestra.run.smithi099.stderr:You should consider upgrading via the 'pip install --upgrade pip' command. 2023-09-06T18:40:52.640 INFO:teuthology.orchestra.run.smithi099.stdout:Requirement already satisfied: nose>=1.0.0 in /usr/lib/python3.6/site-packages (from -r requirements.txt (line 1)) (1.3.7) 2023-09-06T18:40:52.889 INFO:teuthology.orchestra.run.smithi099.stdout:Collecting boto>=2.6.0 2023-09-06T18:40:52.947 INFO:teuthology.orchestra.run.smithi099.stdout: Downloading boto-2.49.0-py2.py3-none-any.whl (1.4 MB) 2023-09-06T18:40:54.011 INFO:teuthology.orchestra.run.smithi099.stdout:Collecting boto3>=1.0.0 2023-09-06T18:40:54.030 INFO:teuthology.orchestra.run.smithi099.stdout: Downloading boto3-1.23.10-py3-none-any.whl (132 kB) 2023-09-06T18:40:55.005 INFO:teuthology.orchestra.run.smithi099.stdout:Collecting configparser>=5.0.0 2023-09-06T18:40:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: Downloading configparser-5.2.0-py3-none-any.whl (19 kB) 2023-09-06T18:40:55.465 INFO:teuthology.orchestra.run.smithi099.stdout:Collecting kafka-python>=2.0.0 2023-09-06T18:40:55.491 INFO:teuthology.orchestra.run.smithi099.stdout: Downloading kafka_python-2.0.2-py2.py3-none-any.whl (246 kB) 2023-09-06T18:40:55.527 INFO:teuthology.orchestra.run.smithi099.stdout:Requirement already satisfied: pika in /usr/lib/python3.6/site-packages (from -r requirements.txt (line 6)) (1.0.1) 2023-09-06T18:40:55.601 INFO:teuthology.orchestra.run.smithi099.stdout:Collecting pika 2023-09-06T18:40:55.620 INFO:teuthology.orchestra.run.smithi099.stdout: Downloading pika-1.3.1-py3-none-any.whl (155 kB) 2023-09-06T18:40:55.711 INFO:teuthology.orchestra.run.smithi099.stdout:Collecting cloudevents 2023-09-06T18:40:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: Downloading cloudevents-1.9.0-py3-none-any.whl (47 kB) 2023-09-06T18:40:55.770 INFO:teuthology.orchestra.run.smithi099.stdout:Requirement already satisfied: xmltodict in /usr/lib/python3.6/site-packages (from -r requirements.txt (line 8)) (0.12.0) 2023-09-06T18:40:55.816 INFO:teuthology.orchestra.run.smithi099.stdout:Collecting xmltodict 2023-09-06T18:40:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: Downloading xmltodict-0.13.0-py2.py3-none-any.whl (10.0 kB) 2023-09-06T18:40:56.819 INFO:teuthology.orchestra.run.smithi099.stdout:Collecting botocore<1.27.0,>=1.26.10 2023-09-06T18:40:56.840 INFO:teuthology.orchestra.run.smithi099.stdout: Downloading botocore-1.26.10-py3-none-any.whl (8.8 MB) 2023-09-06T18:40:57.104 INFO:teuthology.orchestra.run.smithi099.stdout:Collecting jmespath<2.0.0,>=0.7.1 2023-09-06T18:40:57.126 INFO:teuthology.orchestra.run.smithi099.stdout: Downloading jmespath-0.10.0-py2.py3-none-any.whl (24 kB) 2023-09-06T18:40:57.228 INFO:teuthology.orchestra.run.smithi099.stdout:Collecting s3transfer<0.6.0,>=0.5.0 2023-09-06T18:40:57.246 INFO:teuthology.orchestra.run.smithi099.stdout: Downloading s3transfer-0.5.2-py3-none-any.whl (79 kB) 2023-09-06T18:40:57.353 INFO:teuthology.orchestra.run.smithi099.stdout:Collecting deprecation<3.0,>=2.0 2023-09-06T18:40:57.371 INFO:teuthology.orchestra.run.smithi099.stdout: Downloading deprecation-2.1.0-py2.py3-none-any.whl (11 kB) 2023-09-06T18:40:57.512 INFO:teuthology.orchestra.run.smithi099.stdout:Collecting urllib3<1.27,>=1.25.4 2023-09-06T18:40:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: Downloading urllib3-1.26.16-py2.py3-none-any.whl (143 kB) 2023-09-06T18:40:57.576 INFO:teuthology.orchestra.run.smithi099.stdout:Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/lib/python3.6/site-packages (from botocore<1.27.0,>=1.26.10->boto3>=1.0.0->-r requirements.txt (line 3)) (2.6.1) 2023-09-06T18:40:57.685 INFO:teuthology.orchestra.run.smithi099.stdout:Collecting packaging 2023-09-06T18:40:57.703 INFO:teuthology.orchestra.run.smithi099.stdout: Downloading packaging-21.3-py3-none-any.whl (40 kB) 2023-09-06T18:40:57.913 INFO:teuthology.orchestra.run.smithi099.stdout:Requirement already satisfied: six>=1.5 in /usr/lib/python3.6/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.27.0,>=1.26.10->boto3>=1.0.0->-r requirements.txt (line 3)) (1.11.0) 2023-09-06T18:40:57.933 INFO:teuthology.orchestra.run.smithi099.stdout:Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/lib/python3.6/site-packages (from packaging->deprecation<3.0,>=2.0->cloudevents->-r requirements.txt (line 7)) (2.1.10) 2023-09-06T18:40:58.174 INFO:teuthology.orchestra.run.smithi099.stdout:Installing collected packages: urllib3, jmespath, packaging, botocore, s3transfer, deprecation, xmltodict, pika, kafka-python, configparser, cloudevents, boto3, boto 2023-09-06T18:40:58.174 INFO:teuthology.orchestra.run.smithi099.stdout: Attempting uninstall: urllib3 2023-09-06T18:40:58.175 INFO:teuthology.orchestra.run.smithi099.stdout: Found existing installation: urllib3 1.24.2 2023-09-06T18:40:58.175 INFO:teuthology.orchestra.run.smithi099.stdout: Not uninstalling urllib3 at /usr/lib/python3.6/site-packages, outside environment /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv 2023-09-06T18:40:58.175 INFO:teuthology.orchestra.run.smithi099.stdout: Can't uninstall 'urllib3'. No files were found to uninstall. 2023-09-06T18:40:59.089 INFO:teuthology.orchestra.run.smithi099.stdout: Attempting uninstall: xmltodict 2023-09-06T18:40:59.090 INFO:teuthology.orchestra.run.smithi099.stdout: Found existing installation: xmltodict 0.12.0 2023-09-06T18:40:59.090 INFO:teuthology.orchestra.run.smithi099.stdout: Not uninstalling xmltodict at /usr/lib/python3.6/site-packages, outside environment /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv 2023-09-06T18:40:59.090 INFO:teuthology.orchestra.run.smithi099.stdout: Can't uninstall 'xmltodict'. No files were found to uninstall. 2023-09-06T18:40:59.140 INFO:teuthology.orchestra.run.smithi099.stdout: Attempting uninstall: pika 2023-09-06T18:40:59.140 INFO:teuthology.orchestra.run.smithi099.stdout: Found existing installation: pika 1.0.1 2023-09-06T18:40:59.141 INFO:teuthology.orchestra.run.smithi099.stdout: Not uninstalling pika at /usr/lib/python3.6/site-packages, outside environment /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv 2023-09-06T18:40:59.141 INFO:teuthology.orchestra.run.smithi099.stdout: Can't uninstall 'pika'. No files were found to uninstall. 2023-09-06T18:41:00.146 INFO:teuthology.orchestra.run.smithi099.stderr:ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2023-09-06T18:41:00.147 INFO:teuthology.orchestra.run.smithi099.stderr:requests 2.20.0 requires urllib3<1.25,>=1.21.1, but you have urllib3 1.26.16 which is incompatible. 2023-09-06T18:41:00.147 INFO:teuthology.orchestra.run.smithi099.stdout:Successfully installed boto-2.49.0 boto3-1.23.10 botocore-1.26.10 cloudevents-1.9.0 configparser-5.2.0 deprecation-2.1.0 jmespath-0.10.0 kafka-python-2.0.2 packaging-21.3 pika-1.3.1 s3transfer-0.5.2 urllib3-1.26.16 xmltodict-0.13.0 2023-09-06T18:41:00.678 INFO:teuthology.orchestra.run.smithi099.stdout:running develop 2023-09-06T18:41:00.764 INFO:teuthology.orchestra.run.smithi099.stdout:running egg_info 2023-09-06T18:41:00.764 INFO:teuthology.orchestra.run.smithi099.stdout:creating bn_tests.egg-info 2023-09-06T18:41:00.764 INFO:teuthology.orchestra.run.smithi099.stdout:writing bn_tests.egg-info/PKG-INFO 2023-09-06T18:41:00.764 INFO:teuthology.orchestra.run.smithi099.stdout:writing dependency_links to bn_tests.egg-info/dependency_links.txt 2023-09-06T18:41:00.765 INFO:teuthology.orchestra.run.smithi099.stdout:writing requirements to bn_tests.egg-info/requires.txt 2023-09-06T18:41:00.765 INFO:teuthology.orchestra.run.smithi099.stdout:writing top-level names to bn_tests.egg-info/top_level.txt 2023-09-06T18:41:00.765 INFO:teuthology.orchestra.run.smithi099.stdout:writing manifest file 'bn_tests.egg-info/SOURCES.txt' 2023-09-06T18:41:00.766 INFO:teuthology.orchestra.run.smithi099.stdout:reading manifest file 'bn_tests.egg-info/SOURCES.txt' 2023-09-06T18:41:00.766 INFO:teuthology.orchestra.run.smithi099.stdout:writing manifest file 'bn_tests.egg-info/SOURCES.txt' 2023-09-06T18:41:00.782 INFO:teuthology.orchestra.run.smithi099.stdout:running build_ext 2023-09-06T18:41:00.791 INFO:teuthology.orchestra.run.smithi099.stdout:Creating /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib/python3.6/site-packages/bn-tests.egg-link (link to .) 2023-09-06T18:41:00.791 INFO:teuthology.orchestra.run.smithi099.stdout:Adding bn-tests 0.0.1 to easy-install.pth file 2023-09-06T18:41:00.792 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:41:00.792 INFO:teuthology.orchestra.run.smithi099.stdout:Installed /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification 2023-09-06T18:41:00.793 INFO:teuthology.orchestra.run.smithi099.stdout:Processing dependencies for bn-tests==0.0.1 2023-09-06T18:41:00.815 INFO:teuthology.orchestra.run.smithi099.stdout:Searching for boto3==1.23.10 2023-09-06T18:41:00.815 INFO:teuthology.orchestra.run.smithi099.stdout:Best match: boto3 1.23.10 2023-09-06T18:41:00.816 INFO:teuthology.orchestra.run.smithi099.stdout:Adding boto3 1.23.10 to easy-install.pth file 2023-09-06T18:41:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:41:00.816 INFO:teuthology.orchestra.run.smithi099.stdout:Using /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib/python3.6/site-packages 2023-09-06T18:41:00.817 INFO:teuthology.orchestra.run.smithi099.stdout:Searching for boto==2.49.0 2023-09-06T18:41:00.817 INFO:teuthology.orchestra.run.smithi099.stdout:Best match: boto 2.49.0 2023-09-06T18:41:00.818 INFO:teuthology.orchestra.run.smithi099.stdout:Adding boto 2.49.0 to easy-install.pth file 2023-09-06T18:41:00.818 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:41:00.818 INFO:teuthology.orchestra.run.smithi099.stdout:Using /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib/python3.6/site-packages 2023-09-06T18:41:00.818 INFO:teuthology.orchestra.run.smithi099.stdout:Searching for botocore==1.26.10 2023-09-06T18:41:00.819 INFO:teuthology.orchestra.run.smithi099.stdout:Best match: botocore 1.26.10 2023-09-06T18:41:00.819 INFO:teuthology.orchestra.run.smithi099.stdout:Adding botocore 1.26.10 to easy-install.pth file 2023-09-06T18:41:00.819 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:41:00.819 INFO:teuthology.orchestra.run.smithi099.stdout:Using /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib/python3.6/site-packages 2023-09-06T18:41:00.819 INFO:teuthology.orchestra.run.smithi099.stdout:Searching for s3transfer==0.5.2 2023-09-06T18:41:00.819 INFO:teuthology.orchestra.run.smithi099.stdout:Best match: s3transfer 0.5.2 2023-09-06T18:41:00.821 INFO:teuthology.orchestra.run.smithi099.stdout:Adding s3transfer 0.5.2 to easy-install.pth file 2023-09-06T18:41:00.821 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:41:00.821 INFO:teuthology.orchestra.run.smithi099.stdout:Using /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib/python3.6/site-packages 2023-09-06T18:41:00.822 INFO:teuthology.orchestra.run.smithi099.stdout:Searching for jmespath==0.10.0 2023-09-06T18:41:00.822 INFO:teuthology.orchestra.run.smithi099.stdout:Best match: jmespath 0.10.0 2023-09-06T18:41:00.822 INFO:teuthology.orchestra.run.smithi099.stdout:Adding jmespath 0.10.0 to easy-install.pth file 2023-09-06T18:41:00.822 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:41:00.822 INFO:teuthology.orchestra.run.smithi099.stdout:Using /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib/python3.6/site-packages 2023-09-06T18:41:00.823 INFO:teuthology.orchestra.run.smithi099.stdout:Searching for urllib3==1.26.16 2023-09-06T18:41:00.823 INFO:teuthology.orchestra.run.smithi099.stdout:Best match: urllib3 1.26.16 2023-09-06T18:41:00.823 INFO:teuthology.orchestra.run.smithi099.stdout:Adding urllib3 1.26.16 to easy-install.pth file 2023-09-06T18:41:00.823 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:41:00.823 INFO:teuthology.orchestra.run.smithi099.stdout:Using /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib/python3.6/site-packages 2023-09-06T18:41:00.823 INFO:teuthology.orchestra.run.smithi099.stdout:Searching for python-dateutil==2.6.1 2023-09-06T18:41:00.824 INFO:teuthology.orchestra.run.smithi099.stdout:Best match: python-dateutil 2.6.1 2023-09-06T18:41:00.824 INFO:teuthology.orchestra.run.smithi099.stdout:Adding python-dateutil 2.6.1 to easy-install.pth file 2023-09-06T18:41:00.824 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:41:00.824 INFO:teuthology.orchestra.run.smithi099.stdout:Using /usr/lib/python3.6/site-packages 2023-09-06T18:41:00.837 INFO:teuthology.orchestra.run.smithi099.stdout:Searching for six==1.11.0 2023-09-06T18:41:00.837 INFO:teuthology.orchestra.run.smithi099.stdout:Best match: six 1.11.0 2023-09-06T18:41:00.837 INFO:teuthology.orchestra.run.smithi099.stdout:Adding six 1.11.0 to easy-install.pth file 2023-09-06T18:41:00.837 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:41:00.838 INFO:teuthology.orchestra.run.smithi099.stdout:Using /usr/lib/python3.6/site-packages 2023-09-06T18:41:00.838 INFO:teuthology.orchestra.run.smithi099.stdout:Finished processing dependencies for bn-tests==0.0.1 2023-09-06T18:41:00.857 INFO:tasks.notification_tests:Running bucket-notifications-tests... 2023-09-06T18:41:00.858 DEBUG:teuthology.orchestra.run.smithi099:bucket notification tests against different endpoints> BNTESTS_CONF=/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/bn-tests.client.0.conf /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/bin/python -m nose -s /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py -v -a amqp_test 2023-09-06T18:41:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:test s3 notification filter on master ... /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto3/compat.py:88: PythonDeprecationWarning: Boto3 will no longer support Python 3.6 starting May 30, 2022. To continue receiving service updates, bug fixes, and security updates please upgrade to Python 3.7 or later. More information can be found here: https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/ 2023-09-06T18:41:10.671 INFO:teuthology.orchestra.run.smithi099.stderr: warnings.warn(warning, PythonDeprecationWarning) 2023-09-06T18:41:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:ok 2023-09-06T18:41:12.644 INFO:tasks.rgw.client.0.smithi099.stdout:*** Caught signal (Segmentation fault) ** 2023-09-06T18:41:12.645 INFO:tasks.rgw.client.0.smithi099.stdout: in thread 7f5c0191a700 thread_name:msgr-worker-1 2023-09-06T18:41:12.646 INFO:tasks.rgw.client.0.smithi099.stdout: ceph version 18.0.0-5801-g5535a92b (5535a92bd618b7e580a2fce32e9dfab6620d1086) reef (dev) 2023-09-06T18:41:12.646 INFO:tasks.rgw.client.0.smithi099.stdout: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f5c09be1cf0] 2023-09-06T18:41:12.646 INFO:tasks.rgw.client.0.smithi099.stdout: 2: /lib64/librados.so.2(+0x113293) [0x7f5c0e51d293] 2023-09-06T18:41:12.646 INFO:tasks.rgw.client.0.smithi099.stdout: 3: /lib64/librados.so.2(+0x11478b) [0x7f5c0e51e78b] 2023-09-06T18:41:12.647 INFO:tasks.rgw.client.0.smithi099.stdout: 4: /lib64/librados.so.2(+0x117966) [0x7f5c0e521966] 2023-09-06T18:41:12.647 INFO:tasks.rgw.client.0.smithi099.stdout: 5: (DispatchQueue::fast_dispatch(boost::intrusive_ptr const&)+0x23) [0x7f5c0cccba13] 2023-09-06T18:41:12.647 INFO:tasks.rgw.client.0.smithi099.stdout: 6: (ProtocolV2::handle_message()+0xee6) [0x7f5c0cdc3046] 2023-09-06T18:41:12.647 INFO:tasks.rgw.client.0.smithi099.stdout: 7: (ProtocolV2::run_continuation(Ct&)+0x39) [0x7f5c0cdbee19] 2023-09-06T18:41:12.647 INFO:tasks.rgw.client.0.smithi099.stdout: 8: (AsyncConnection::process()+0x8fd) [0x7f5c0cd86dfd] 2023-09-06T18:41:12.647 INFO:tasks.rgw.client.0.smithi099.stdout: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0x1d4) [0x7f5c0cde0154] 2023-09-06T18:41:12.648 INFO:tasks.rgw.client.0.smithi099.stdout: 10: /usr/lib64/ceph/libceph-common.so.2(+0x646bc6) [0x7f5c0cde8bc6] 2023-09-06T18:41:12.648 INFO:tasks.rgw.client.0.smithi099.stdout: 11: /lib64/libstdc++.so.6(+0xc2b13) [0x7f5c09359b13] 2023-09-06T18:41:12.648 INFO:tasks.rgw.client.0.smithi099.stdout: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f5c09bd71ca] 2023-09-06T18:41:12.648 INFO:tasks.rgw.client.0.smithi099.stdout: 13: clone() 2023-09-06T18:41:12.648 INFO:tasks.rgw.client.0.smithi099.stdout:2023-09-06T18:41:12.644+0000 7f5c0191a700 -1 *** Caught signal (Segmentation fault) ** 2023-09-06T18:41:12.649 INFO:tasks.rgw.client.0.smithi099.stdout: in thread 7f5c0191a700 thread_name:msgr-worker-1 2023-09-06T18:41:12.649 INFO:tasks.rgw.client.0.smithi099.stdout: 2023-09-06T18:41:12.649 INFO:tasks.rgw.client.0.smithi099.stdout: ceph version 18.0.0-5801-g5535a92b (5535a92bd618b7e580a2fce32e9dfab6620d1086) reef (dev) 2023-09-06T18:41:12.649 INFO:tasks.rgw.client.0.smithi099.stdout: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f5c09be1cf0] 2023-09-06T18:41:12.650 INFO:tasks.rgw.client.0.smithi099.stdout: 2: /lib64/librados.so.2(+0x113293) [0x7f5c0e51d293] 2023-09-06T18:41:12.650 INFO:tasks.rgw.client.0.smithi099.stdout: 3: /lib64/librados.so.2(+0x11478b) [0x7f5c0e51e78b] 2023-09-06T18:41:12.650 INFO:tasks.rgw.client.0.smithi099.stdout: 4: /lib64/librados.so.2(+0x117966) [0x7f5c0e521966] 2023-09-06T18:41:12.650 INFO:tasks.rgw.client.0.smithi099.stdout: 5: (DispatchQueue::fast_dispatch(boost::intrusive_ptr const&)+0x23) [0x7f5c0cccba13] 2023-09-06T18:41:12.650 INFO:tasks.rgw.client.0.smithi099.stdout: 6: (ProtocolV2::handle_message()+0xee6) [0x7f5c0cdc3046] 2023-09-06T18:41:12.651 INFO:tasks.rgw.client.0.smithi099.stdout: 7: (ProtocolV2::run_continuation(Ct&)+0x39) [0x7f5c0cdbee19] 2023-09-06T18:41:12.651 INFO:tasks.rgw.client.0.smithi099.stdout: 8: (AsyncConnection::process()+0x8fd) [0x7f5c0cd86dfd] 2023-09-06T18:41:12.651 INFO:tasks.rgw.client.0.smithi099.stdout: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0x1d4) [0x7f5c0cde0154] 2023-09-06T18:41:12.651 INFO:tasks.rgw.client.0.smithi099.stdout: 10: /usr/lib64/ceph/libceph-common.so.2(+0x646bc6) [0x7f5c0cde8bc6] 2023-09-06T18:41:12.651 INFO:tasks.rgw.client.0.smithi099.stdout: 11: /lib64/libstdc++.so.6(+0xc2b13) [0x7f5c09359b13] 2023-09-06T18:41:12.652 INFO:tasks.rgw.client.0.smithi099.stdout: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f5c09bd71ca] 2023-09-06T18:41:12.652 INFO:tasks.rgw.client.0.smithi099.stdout: 13: clone() 2023-09-06T18:41:12.652 INFO:tasks.rgw.client.0.smithi099.stdout: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-06T18:41:12.652 INFO:tasks.rgw.client.0.smithi099.stdout: 2023-09-06T18:41:12.663 INFO:tasks.rgw.client.0.smithi099.stdout: 0> 2023-09-06T18:41:12.644+0000 7f5c0191a700 -1 *** Caught signal (Segmentation fault) ** 2023-09-06T18:41:12.663 INFO:tasks.rgw.client.0.smithi099.stdout: in thread 7f5c0191a700 thread_name:msgr-worker-1 2023-09-06T18:41:12.664 INFO:tasks.rgw.client.0.smithi099.stdout: 2023-09-06T18:41:12.664 INFO:tasks.rgw.client.0.smithi099.stdout: ceph version 18.0.0-5801-g5535a92b (5535a92bd618b7e580a2fce32e9dfab6620d1086) reef (dev) 2023-09-06T18:41:12.664 INFO:tasks.rgw.client.0.smithi099.stdout: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f5c09be1cf0] 2023-09-06T18:41:12.664 INFO:tasks.rgw.client.0.smithi099.stdout: 2: /lib64/librados.so.2(+0x113293) [0x7f5c0e51d293] 2023-09-06T18:41:12.664 INFO:tasks.rgw.client.0.smithi099.stdout: 3: /lib64/librados.so.2(+0x11478b) [0x7f5c0e51e78b] 2023-09-06T18:41:12.665 INFO:tasks.rgw.client.0.smithi099.stdout: 4: /lib64/librados.so.2(+0x117966) [0x7f5c0e521966] 2023-09-06T18:41:12.665 INFO:tasks.rgw.client.0.smithi099.stdout: 5: (DispatchQueue::fast_dispatch(boost::intrusive_ptr const&)+0x23) [0x7f5c0cccba13] 2023-09-06T18:41:12.665 INFO:tasks.rgw.client.0.smithi099.stdout: 6: (ProtocolV2::handle_message()+0xee6) [0x7f5c0cdc3046] 2023-09-06T18:41:12.665 INFO:tasks.rgw.client.0.smithi099.stdout: 7: (ProtocolV2::run_continuation(Ct&)+0x39) [0x7f5c0cdbee19] 2023-09-06T18:41:12.666 INFO:tasks.rgw.client.0.smithi099.stdout: 8: (AsyncConnection::process()+0x8fd) [0x7f5c0cd86dfd] 2023-09-06T18:41:12.666 INFO:tasks.rgw.client.0.smithi099.stdout: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0x1d4) [0x7f5c0cde0154] 2023-09-06T18:41:12.666 INFO:tasks.rgw.client.0.smithi099.stdout: 10: /usr/lib64/ceph/libceph-common.so.2(+0x646bc6) [0x7f5c0cde8bc6] 2023-09-06T18:41:12.666 INFO:tasks.rgw.client.0.smithi099.stdout: 11: /lib64/libstdc++.so.6(+0xc2b13) [0x7f5c09359b13] 2023-09-06T18:41:12.666 INFO:tasks.rgw.client.0.smithi099.stdout: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f5c09bd71ca] 2023-09-06T18:41:12.667 INFO:tasks.rgw.client.0.smithi099.stdout: 13: clone() 2023-09-06T18:41:12.667 INFO:tasks.rgw.client.0.smithi099.stdout: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-06T18:41:12.667 INFO:tasks.rgw.client.0.smithi099.stdout: 2023-09-06T18:41:12.679 INFO:tasks.rgw.client.0.smithi099.stdout: -50> 2023-09-06T18:41:12.644+0000 7f5c0191a700 -1 *** Caught signal (Segmentation fault) ** 2023-09-06T18:41:12.679 INFO:tasks.rgw.client.0.smithi099.stdout: in thread 7f5c0191a700 thread_name:msgr-worker-1 2023-09-06T18:41:12.679 INFO:tasks.rgw.client.0.smithi099.stdout: 2023-09-06T18:41:12.679 INFO:tasks.rgw.client.0.smithi099.stdout: ceph version 18.0.0-5801-g5535a92b (5535a92bd618b7e580a2fce32e9dfab6620d1086) reef (dev) 2023-09-06T18:41:12.680 INFO:tasks.rgw.client.0.smithi099.stdout: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f5c09be1cf0] 2023-09-06T18:41:12.680 INFO:tasks.rgw.client.0.smithi099.stdout: 2: /lib64/librados.so.2(+0x113293) [0x7f5c0e51d293] 2023-09-06T18:41:12.680 INFO:tasks.rgw.client.0.smithi099.stdout: 3: /lib64/librados.so.2(+0x11478b) [0x7f5c0e51e78b] 2023-09-06T18:41:12.680 INFO:tasks.rgw.client.0.smithi099.stdout: 4: /lib64/librados.so.2(+0x117966) [0x7f5c0e521966] 2023-09-06T18:41:12.681 INFO:tasks.rgw.client.0.smithi099.stdout: 5: (DispatchQueue::fast_dispatch(boost::intrusive_ptr const&)+0x23) [0x7f5c0cccba13] 2023-09-06T18:41:12.681 INFO:tasks.rgw.client.0.smithi099.stdout: 6: (ProtocolV2::handle_message()+0xee6) [0x7f5c0cdc3046] 2023-09-06T18:41:12.681 INFO:tasks.rgw.client.0.smithi099.stdout: 7: (ProtocolV2::run_continuation(Ct&)+0x39) [0x7f5c0cdbee19] 2023-09-06T18:41:12.681 INFO:tasks.rgw.client.0.smithi099.stdout: 8: (AsyncConnection::process()+0x8fd) [0x7f5c0cd86dfd] 2023-09-06T18:41:12.682 INFO:tasks.rgw.client.0.smithi099.stdout: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0x1d4) [0x7f5c0cde0154] 2023-09-06T18:41:12.682 INFO:tasks.rgw.client.0.smithi099.stdout: 10: /usr/lib64/ceph/libceph-common.so.2(+0x646bc6) [0x7f5c0cde8bc6] 2023-09-06T18:41:12.682 INFO:tasks.rgw.client.0.smithi099.stdout: 11: /lib64/libstdc++.so.6(+0xc2b13) [0x7f5c09359b13] 2023-09-06T18:41:12.682 INFO:tasks.rgw.client.0.smithi099.stdout: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f5c09bd71ca] 2023-09-06T18:41:12.683 INFO:tasks.rgw.client.0.smithi099.stdout: 13: clone() 2023-09-06T18:41:12.683 INFO:tasks.rgw.client.0.smithi099.stdout: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-06T18:41:12.683 INFO:tasks.rgw.client.0.smithi099.stdout: 2023-09-06T18:41:12.874 INFO:tasks.rgw.client.0.smithi099.stderr:daemon-helper: command crashed with signal 11 2023-09-06T18:41:17.242 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s 2023-09-06T18:41:23.551 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~6s 2023-09-06T18:41:29.861 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~13s 2023-09-06T18:41:36.171 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~19s 2023-09-06T18:41:42.482 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~25s 2023-09-06T18:41:48.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~32s 2023-09-06T18:41:55.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~38s 2023-09-06T18:42:01.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~44s 2023-09-06T18:42:07.722 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~50s 2023-09-06T18:42:14.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~57s 2023-09-06T18:42:20.345 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~63s 2023-09-06T18:42:26.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~69s 2023-09-06T18:42:32.966 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~76s 2023-09-06T18:42:39.277 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~82s 2023-09-06T18:42:45.587 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~88s 2023-09-06T18:42:51.897 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~95s 2023-09-06T18:42:58.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~101s 2023-09-06T18:43:04.520 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~107s 2023-09-06T18:43:10.830 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~114s 2023-09-06T18:43:17.139 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~120s 2023-09-06T18:43:23.450 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~126s 2023-09-06T18:43:29.759 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~133s 2023-09-06T18:43:36.069 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~139s 2023-09-06T18:43:42.378 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~145s 2023-09-06T18:43:48.687 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~151s 2023-09-06T18:43:54.997 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~158s 2023-09-06T18:44:01.308 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~164s 2023-09-06T18:44:07.617 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~170s 2023-09-06T18:44:13.513 INFO:teuthology.orchestra.run.smithi099.stderr:test pushing amqp s3 notification on master ... ERROR 2023-09-06T18:44:13.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~177s 2023-09-06T18:44:20.236 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~183s 2023-09-06T18:44:26.547 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~189s 2023-09-06T18:44:32.857 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~196s 2023-09-06T18:44:39.166 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~202s 2023-09-06T18:44:45.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~208s 2023-09-06T18:44:51.787 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~215s 2023-09-06T18:44:58.098 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~221s 2023-09-06T18:45:04.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~227s 2023-09-06T18:45:10.720 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~233s 2023-09-06T18:45:17.030 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~240s 2023-09-06T18:45:23.340 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~246s 2023-09-06T18:45:29.650 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~252s 2023-09-06T18:45:35.961 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~259s 2023-09-06T18:45:38.958 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn.test_ps_s3_creation_triggers_on_master ... ERROR 2023-09-06T18:45:42.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~265s 2023-09-06T18:45:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:test that uploads object using HTTP POST ... /usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.16) or chardet (3.0.4) doesn't match a supported version! 2023-09-06T18:45:47.926 INFO:teuthology.orchestra.run.smithi099.stderr: RequestsDependencyWarning) 2023-09-06T18:45:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto3/compat.py:88: PythonDeprecationWarning: Boto3 will no longer support Python 3.6 starting May 30, 2022. To continue receiving service updates, bug fixes, and security updates please upgrade to Python 3.7 or later. More information can be found here: https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/ 2023-09-06T18:45:47.926 INFO:teuthology.orchestra.run.smithi099.stderr: warnings.warn(warning, PythonDeprecationWarning) 2023-09-06T18:45:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:ERROR 2023-09-06T18:45:48.581 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~271s 2023-09-06T18:45:54.892 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~278s 2023-09-06T18:46:01.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~284s 2023-09-06T18:46:07.509 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~290s 2023-09-06T18:46:13.819 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~297s 2023-09-06T18:46:20.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~303s 2023-09-06T18:46:20.128 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-09-06T18:46:21.436 INFO:tasks.ceph.osd.0:Sent signal 15 2023-09-06T18:46:21.437 INFO:tasks.ceph.osd.1:Sent signal 15 2023-09-06T18:46:21.438 INFO:tasks.ceph.osd.2:Sent signal 15 2023-09-06T18:46:21.438 INFO:tasks.ceph.osd.3:Sent signal 15 2023-09-06T18:46:21.439 INFO:tasks.ceph.osd.4:Sent signal 15 2023-09-06T18:46:21.439 INFO:tasks.ceph.osd.5:Sent signal 15 2023-09-06T18:46:21.439 INFO:tasks.ceph.osd.6:Sent signal 15 2023-09-06T18:46:21.440 INFO:tasks.ceph.osd.7:Sent signal 15 2023-09-06T18:46:21.440 INFO:tasks.ceph.mon.a:Sent signal 15 2023-09-06T18:46:21.441 INFO:tasks.ceph.mon.c:Sent signal 15 2023-09-06T18:46:21.441 INFO:tasks.ceph.mon.b:Sent signal 15 2023-09-06T18:46:21.442 INFO:tasks.ceph.mgr.y:Sent signal 15 2023-09-06T18:46:21.442 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-09-06T18:46:21.443 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-09-06T18:46:21.439+0000 7eff1a171700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 115827) UID: 0 2023-09-06T18:46:21.443 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-09-06T18:46:21.439+0000 7eff1a171700 -1 osd.4 34 *** Got signal Terminated *** 2023-09-06T18:46:21.444 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-09-06T18:46:21.439+0000 7eff1a171700 -1 osd.4 34 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-06T18:46:21.444 INFO:tasks.ceph.osd.0.smithi099.stderr:2023-09-06T18:46:21.436+0000 7fb97921d700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 115878) UID: 0 2023-09-06T18:46:21.444 INFO:tasks.ceph.osd.0.smithi099.stderr:2023-09-06T18:46:21.436+0000 7fb97921d700 -1 osd.0 34 *** Got signal Terminated *** 2023-09-06T18:46:21.444 INFO:tasks.ceph.osd.0.smithi099.stderr:2023-09-06T18:46:21.436+0000 7fb97921d700 -1 osd.0 34 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-06T18:46:21.445 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-09-06T18:46:21.439+0000 7ffb06eed700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 115834) UID: 0 2023-09-06T18:46:21.445 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-09-06T18:46:21.439+0000 7ffb06eed700 -1 osd.6 34 *** Got signal Terminated *** 2023-09-06T18:46:21.445 INFO:tasks.ceph.osd.1.smithi099.stderr:2023-09-06T18:46:21.437+0000 7f419efc7700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 115908) UID: 0 2023-09-06T18:46:21.446 INFO:tasks.ceph.osd.1.smithi099.stderr:2023-09-06T18:46:21.437+0000 7f419efc7700 -1 osd.1 34 *** Got signal Terminated *** 2023-09-06T18:46:21.446 INFO:tasks.ceph.osd.1.smithi099.stderr:2023-09-06T18:46:21.437+0000 7f419efc7700 -1 osd.1 34 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-06T18:46:21.446 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-09-06T18:46:21.439+0000 7fe8ba7e7700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 115830) UID: 0 2023-09-06T18:46:21.447 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-09-06T18:46:21.439+0000 7fe8ba7e7700 -1 osd.5 34 *** Got signal Terminated *** 2023-09-06T18:46:21.447 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-09-06T18:46:21.439+0000 7fe8ba7e7700 -1 osd.5 34 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-06T18:46:21.447 INFO:tasks.ceph.osd.2.smithi099.stderr:2023-09-06T18:46:21.437+0000 7f470a193700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 115909) UID: 0 2023-09-06T18:46:21.447 INFO:tasks.ceph.osd.2.smithi099.stderr:2023-09-06T18:46:21.437+0000 7f470a193700 -1 osd.2 34 *** Got signal Terminated *** 2023-09-06T18:46:21.447 INFO:tasks.ceph.osd.2.smithi099.stderr:2023-09-06T18:46:21.437+0000 7f470a193700 -1 osd.2 34 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-06T18:46:21.448 INFO:tasks.ceph.osd.3.smithi099.stderr:2023-09-06T18:46:21.437+0000 7fbc6d5bb700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 115910) UID: 0 2023-09-06T18:46:21.448 INFO:tasks.ceph.osd.3.smithi099.stderr:2023-09-06T18:46:21.437+0000 7fbc6d5bb700 -1 osd.3 34 *** Got signal Terminated *** 2023-09-06T18:46:21.449 INFO:tasks.ceph.osd.3.smithi099.stderr:2023-09-06T18:46:21.437+0000 7fbc6d5bb700 -1 osd.3 34 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-06T18:46:21.449 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-09-06T18:46:21.439+0000 7ffb06eed700 -1 osd.6 34 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-06T18:46:21.450 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-09-06T18:46:21.439+0000 7f3ed73fe700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 115488) UID: 0 2023-09-06T18:46:21.450 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-09-06T18:46:21.439+0000 7f3ed73fe700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-09-06T18:46:21.450 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-09-06T18:46:21.439+0000 7f1482f77700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 115835) UID: 0 2023-09-06T18:46:21.450 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-09-06T18:46:21.439+0000 7f1482f77700 -1 osd.7 34 *** Got signal Terminated *** 2023-09-06T18:46:21.451 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-09-06T18:46:21.439+0000 7f1482f77700 -1 osd.7 34 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-06T18:46:21.451 INFO:tasks.ceph.mon.c.smithi099.stderr:2023-09-06T18:46:21.440+0000 7fea8d8e0700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 115490) UID: 0 2023-09-06T18:46:21.451 INFO:tasks.ceph.mon.c.smithi099.stderr:2023-09-06T18:46:21.440+0000 7fea8d8e0700 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-09-06T18:46:21.452 INFO:tasks.ceph.mon.b.smithi181.stderr:2023-09-06T18:46:21.441+0000 7f207b6f3700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 115169) UID: 0 2023-09-06T18:46:21.452 INFO:tasks.ceph.mon.b.smithi181.stderr:2023-09-06T18:46:21.441+0000 7f207b6f3700 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-09-06T18:46:21.644 INFO:tasks.ceph.mgr.y.smithi099.stderr:daemon-helper: command crashed with signal 15 2023-09-06T18:46:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:test multipart object upload on master ... ERROR 2023-09-06T18:47:26.386 INFO:teuthology.orchestra.run.smithi099.stderr:test s3 notification of metadata on master ... ERROR 2023-09-06T18:48:49.485 INFO:teuthology.orchestra.run.smithi099.stderr:test s3 notification of metadata on master ... ERROR 2023-09-06T18:50:11.905 INFO:teuthology.orchestra.run.smithi099.stderr:test s3 notification of tags on master ... ERROR 2023-09-06T18:51:30.268 INFO:teuthology.orchestra.run.smithi099.stderr:test s3 notification of object versions ... ERROR 2023-09-06T18:52:52.539 INFO:teuthology.orchestra.run.smithi099.stderr:test s3 notification of deletion markers on master ... ERROR 2023-09-06T18:54:01.850 INFO:teuthology.orchestra.run.smithi099.stderr:test pushing persistent notification amqp ... ERROR 2023-09-06T18:54:38.409 INFO:teuthology.orchestra.run.smithi099.stderr:test pushing persistent notification of large notifications ... ERROR 2023-09-06T18:54:38.410 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:====================================================================== 2023-09-06T18:54:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:ERROR: test pushing amqp s3 notification on master 2023-09-06T18:54:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:---------------------------------------------------------------------- 2023-09-06T18:54:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:38.411 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest 2023-09-06T18:54:38.411 INFO:teuthology.orchestra.run.smithi099.stderr: self.test(*self.arg) 2023-09-06T18:54:38.411 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 1364, in test_ps_s3_notification_push_amqp_on_master 2023-09-06T18:54:38.411 INFO:teuthology.orchestra.run.smithi099.stderr: keys = list(bucket.list()) 2023-09-06T18:54:38.412 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/bucketlistresultset.py", line 34, in bucket_lister 2023-09-06T18:54:38.412 INFO:teuthology.orchestra.run.smithi099.stderr: encoding_type=encoding_type) 2023-09-06T18:54:38.412 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/bucket.py", line 474, in get_all_keys 2023-09-06T18:54:38.412 INFO:teuthology.orchestra.run.smithi099.stderr: '', headers, **params) 2023-09-06T18:54:38.413 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/bucket.py", line 400, in _get_all 2023-09-06T18:54:38.413 INFO:teuthology.orchestra.run.smithi099.stderr: query_args=query_args) 2023-09-06T18:54:38.413 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 671, in make_request 2023-09-06T18:54:38.413 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler 2023-09-06T18:54:38.414 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1071, in make_request 2023-09-06T18:54:38.414 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler) 2023-09-06T18:54:38.414 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1030, in _mexe 2023-09-06T18:54:38.414 INFO:teuthology.orchestra.run.smithi099.stderr: raise ex 2023-09-06T18:54:38.415 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 943, in _mexe 2023-09-06T18:54:38.415 INFO:teuthology.orchestra.run.smithi099.stderr: request.body, request.headers) 2023-09-06T18:54:38.415 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:38.415 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:38.415 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:38.416 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:38.416 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:38.416 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:38.416 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1044, in _send_output 2023-09-06T18:54:38.416 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:38.417 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:38.417 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:38.417 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 954, in connect 2023-09-06T18:54:38.417 INFO:teuthology.orchestra.run.smithi099.stderr: (self.host,self.port), self.timeout, self.source_address) 2023-09-06T18:54:38.418 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 724, in create_connection 2023-09-06T18:54:38.418 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:38.418 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 713, in create_connection 2023-09-06T18:54:38.418 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:-------------------- >> begin captured logging << -------------------- 2023-09-06T18:54:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using access key provided by client. 2023-09-06T18:54:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using secret key provided by client. 2023-09-06T18:54:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/ 2023-09-06T18:54:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/ 2023-09-06T18:54:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/ 2023-09-06T18:54:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {} 2023-09-06T18:54:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.421 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:38.422 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/ 2023-09-06T18:54:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:x9GvqZU1HxmfYtILHO7CbrhUYRA= 2023-09-06T18:54:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:x9GvqZU1HxmfYtILHO7CbrhUYRA=', 'Content-Length': '0'} 2023-09-06T18:54:38.423 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('x-amz-request-id', 'tx000008e527f86db1d4f8d-0064f8c7c6-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Content-Length', '0'), ('Date', 'Wed, 06 Sep 2023 18:41:10 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.423 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: Using EPollPoller 2023-09-06T18:54:38.423 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.base_connection: DEBUG: Created default connection workflow 2023-09-06T18:54:38.423 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: Starting AMQP Connection workflow asynchronously. 2023-09-06T18:54:38.423 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1098.128457083 and callback=functools.partial(>, first=True); now=1098.128457083; delay=0 2023-09-06T18:54:38.424 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: Beginning a new AMQP connection workflow cycle; attempts remaining after this: 0 2023-09-06T18:54:38.424 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1098.12854645 and callback=>; now=1098.12854645; delay=0 2023-09-06T18:54:38.424 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: _try_next_config_async: 'localhost':5672 2023-09-06T18:54:38.424 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: add_callback_threadsafe: added callback=> 2023-09-06T18:54:38.424 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: process_timeouts: invoking callback=> 2023-09-06T18:54:38.425 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: Invoking asynchronous getaddrinfo() completion callback; host='localhost' 2023-09-06T18:54:38.425 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: getaddrinfo returned 2 records 2023-09-06T18:54:38.425 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: Attempting to connect using address record (, , 6, '', ('::1', 5672, 0, 0)) 2023-09-06T18:54:38.425 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: INFO: Pika version 1.3.1 connecting to ('::1', 5672, 0, 0) 2023-09-06T18:54:38.425 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: add_callback_threadsafe: added callback=> 2023-09-06T18:54:38.425 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1108.129228063 and callback=>; now=1098.129228063; delay=10.0 2023-09-06T18:54:38.426 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1113.129259297 and callback=>; now=1098.129259297; delay=15.0 2023-09-06T18:54:38.426 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: process_timeouts: invoking callback=> 2023-09-06T18:54:38.426 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:38.426 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) added handler Wr 2023-09-06T18:54:38.427 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Connection-establishment is in progress for . 2023-09-06T18:54:38.427 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: INFO: Socket connected: 2023-09-06T18:54:38.427 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _AsyncSocketConnector._report_completion(None); 2023-09-06T18:54:38.427 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:38.427 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) removed handler 2023-09-06T18:54:38.428 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: remove_timeout: removing timeout with deadline=1108.129228063 and callback=> 2023-09-06T18:54:38.428 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: TCP connection to broker established: . 2023-09-06T18:54:38.428 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _AsyncStreamConnector.start(); 2023-09-06T18:54:38.428 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: add_callback_threadsafe: added callback=> 2023-09-06T18:54:38.428 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: process_timeouts: invoking callback=> 2023-09-06T18:54:38.428 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _AsyncStreamConnector._start_async(); 2023-09-06T18:54:38.429 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _AsyncStreamConnector._linkup() 2023-09-06T18:54:38.429 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: New Connection state: CLOSED (prev=CLOSED) 2023-09-06T18:54:38.429 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.429 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.429 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >>, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.430 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: New Connection state: INIT (prev=CLOSED) 2023-09-06T18:54:38.430 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: Using external connection workflow. 2023-09-06T18:54:38.430 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _AsyncTransportBase.__init__: 2023-09-06T18:54:38.430 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_reader(11, >) 2023-09-06T18:54:38.430 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_reader(11, _) added handler Rd 2023-09-06T18:54:38.431 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _linkup(): created transport 2023-09-06T18:54:38.431 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: New Connection state: PROTOCOL (prev=INIT) 2023-09-06T18:54:38.431 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:38.431 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:38.431 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.432 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _linkup(): introduced transport to protocol ; _StreamingProtocolShim: params=> 2023-09-06T18:54:38.432 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _AsyncStreamConnector._report_completion((, _StreamingProtocolShim: params=>)); 2023-09-06T18:54:38.432 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: INFO: Streaming transport linked up: (, _StreamingProtocolShim: params=>). 2023-09-06T18:54:38.432 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.432 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.433 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.433 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _AsyncStreamConnector._cleanup(False) 2023-09-06T18:54:38.433 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:38.433 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:38.433 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.433 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 0:Connection.Start 2023-09-06T18:54:38.434 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.434 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.434 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.434 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>> for "0:Connection.Start" 2023-09-06T18:54:38.434 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: New Connection state: START (prev=PROTOCOL) 2023-09-06T18:54:38.435 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.435 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:38.435 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:38.435 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.435 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.436 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:38.436 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:38.436 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.436 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 0:Connection.Tune 2023-09-06T18:54:38.437 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.437 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.437 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.437 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>> for "0:Connection.Tune" 2023-09-06T18:54:38.437 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: New Connection state: TUNE (prev=START) 2023-09-06T18:54:38.438 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: Creating a HeartbeatChecker: 60 2023-09-06T18:54:38.438 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: timeout: 60.000000 send_interval: 30.000000 check_interval: 65.000000 2023-09-06T18:54:38.438 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1128.1322728 and callback=>; now=1098.1322728; delay=30.0 2023-09-06T18:54:38.438 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1163.132304859 and callback=>; now=1098.132304859; delay=65 2023-09-06T18:54:38.438 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:38.439 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:38.439 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.439 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.439 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.439 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:38.440 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:38.440 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.440 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 0:Connection.OpenOk 2023-09-06T18:54:38.440 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.440 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.441 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.441 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>> for "0:Connection.OpenOk" 2023-09-06T18:54:38.441 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: New Connection state: OPEN (prev=TUNE) 2023-09-06T18:54:38.441 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 0:_on_connection_open_ok 2023-09-06T18:54:38.441 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling > for "0:_on_connection_open_ok" 2023-09-06T18:54:38.442 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: AMQPConnector: AMQP handshake attempt completed; state=3; error=None; 'localhost'/(, , 6, '', ('::1', 5672, 0, 0)) 2023-09-06T18:54:38.442 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: AMQPConnector: AMQP connection established for 'localhost'/(, , 6, '', ('::1', 5672, 0, 0)): params=> 2023-09-06T18:54:38.442 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: INFO: AMQPConnector - reporting success: params=> 2023-09-06T18:54:38.442 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: remove_timeout: removing timeout with deadline=1113.129259297 and callback=> 2023-09-06T18:54:38.442 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: Connection attempt completed with params=> 2023-09-06T18:54:38.442 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: INFO: AMQPConnectionWorkflow - reporting success: params=> 2023-09-06T18:54:38.443 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.443 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.blocking_connection: INFO: Connection workflow succeeded: params=> 2023-09-06T18:54:38.443 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.443 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: Creating channel 1 2023-09-06T18:54:38.443 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>, 'one_shot': True, 'only': params=>>, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.444 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.444 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.444 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.444 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.444 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:38.445 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:38.445 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.445 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Entering blocking state on frame ; acceptable_replies=[] 2023-09-06T18:54:38.445 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding on_synchronous_complete callback 2023-09-06T18:54:38.445 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.446 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding passed-in RPC response callback 2023-09-06T18:54:38.446 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.446 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>>, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.446 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.446 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>>, 'one_shot': False, 'only': params=>>, 'arguments': None} 2023-09-06T18:54:38.447 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.447 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.blocking_connection: INFO: Created channel=1 2023-09-06T18:54:38.447 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:38.447 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:38.447 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.447 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 1:Channel.OpenOk 2023-09-06T18:54:38.448 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.448 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.448 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.448 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.448 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.448 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.449 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>>> for "1:Channel.OpenOk" 2023-09-06T18:54:38.449 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: 0 blocked frames 2023-09-06T18:54:38.449 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>>> for "1:Channel.OpenOk" 2023-09-06T18:54:38.449 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.449 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:38.450 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:38.450 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.450 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Entering blocking state on frame ; acceptable_replies=[] 2023-09-06T18:54:38.450 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding on_synchronous_complete callback 2023-09-06T18:54:38.451 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.451 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding passed-in RPC response callback 2023-09-06T18:54:38.451 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.451 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:38.451 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:38.452 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.452 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 1:Exchange.DeclareOk 2023-09-06T18:54:38.452 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.452 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.452 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.452 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.453 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.453 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': >, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.453 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>>> for "1:Exchange.DeclareOk" 2023-09-06T18:54:38.453 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: 0 blocked frames 2023-09-06T18:54:38.453 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling > for "1:Exchange.DeclareOk" 2023-09-06T18:54:38.454 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.454 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:38.454 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:38.454 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.454 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Entering blocking state on frame ; acceptable_replies=[] 2023-09-06T18:54:38.455 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding on_synchronous_complete callback 2023-09-06T18:54:38.455 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.455 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding passed-in RPC response callback 2023-09-06T18:54:38.455 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.455 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:38.455 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:38.456 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.456 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 1:Queue.DeclareOk 2023-09-06T18:54:38.456 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.456 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.456 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.457 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.457 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.457 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': >, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.457 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>>> for "1:Queue.DeclareOk" 2023-09-06T18:54:38.457 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: 0 blocked frames 2023-09-06T18:54:38.457 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling > for "1:Queue.DeclareOk" 2023-09-06T18:54:38.458 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.458 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:38.458 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:38.458 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.458 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Entering blocking state on frame ; acceptable_replies=[] 2023-09-06T18:54:38.459 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding on_synchronous_complete callback 2023-09-06T18:54:38.459 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.459 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding passed-in RPC response callback 2023-09-06T18:54:38.459 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.459 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:38.460 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:38.460 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.460 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 1:Queue.BindOk 2023-09-06T18:54:38.460 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': >, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>>> for "1:Queue.BindOk" 2023-09-06T18:54:38.462 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: 0 blocked frames 2023-09-06T18:54:38.462 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling > for "1:Queue.BindOk" 2023-09-06T18:54:38.462 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.462 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:38.462 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Entering blocking state on frame ; acceptable_replies=[(, {'consumer_tag': 'ctag1.b82ef61f102a41d8b2d3143052210335'})] 2023-09-06T18:54:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding on_synchronous_complete callback 2023-09-06T18:54:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': {'consumer_tag': 'ctag1.b82ef61f102a41d8b2d3143052210335'}, 'calls': 1} 2023-09-06T18:54:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding passed-in RPC response callback 2023-09-06T18:54:38.464 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': {'consumer_tag': 'ctag1.b82ef61f102a41d8b2d3143052210335'}, 'calls': 1} 2023-09-06T18:54:38.464 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:38.464 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:38.464 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.464 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 1:Basic.ConsumeOk 2023-09-06T18:54:38.464 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.465 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.465 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Comparing {'consumer_tag': 'ctag1.b82ef61f102a41d8b2d3143052210335'} to {'consumer_tag': 'ctag1.b82ef61f102a41d8b2d3143052210335'} 2023-09-06T18:54:38.465 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #1: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': {'consumer_tag': 'ctag1.b82ef61f102a41d8b2d3143052210335'}, 'calls': 0} 2023-09-06T18:54:38.465 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.465 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.465 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Comparing {'consumer_tag': 'ctag1.b82ef61f102a41d8b2d3143052210335'} to {'consumer_tag': 'ctag1.b82ef61f102a41d8b2d3143052210335'} 2023-09-06T18:54:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #1: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': {'consumer_tag': 'ctag1.b82ef61f102a41d8b2d3143052210335'}, 'calls': 0} 2023-09-06T18:54:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling > for "1:Basic.ConsumeOk" 2023-09-06T18:54:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>>> for "1:Basic.ConsumeOk" 2023-09-06T18:54:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: 0 blocked frames 2023-09-06T18:54:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>>> for "1:Basic.ConsumeOk" 2023-09-06T18:54:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Discarding frame "])> 2023-09-06T18:54:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: Using EPollPoller 2023-09-06T18:54:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.base_connection: DEBUG: Created default connection workflow 2023-09-06T18:54:38.468 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: Starting AMQP Connection workflow asynchronously. 2023-09-06T18:54:38.468 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1098.140204705 and callback=functools.partial(>, first=True); now=1098.140204705; delay=0 2023-09-06T18:54:38.468 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: Beginning a new AMQP connection workflow cycle; attempts remaining after this: 0 2023-09-06T18:54:38.468 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1098.140283595 and callback=>; now=1098.140283595; delay=0 2023-09-06T18:54:38.468 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: _try_next_config_async: 'localhost':5672 2023-09-06T18:54:38.468 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: add_callback_threadsafe: added callback=> 2023-09-06T18:54:38.469 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: process_timeouts: invoking callback=> 2023-09-06T18:54:38.469 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: Invoking asynchronous getaddrinfo() completion callback; host='localhost' 2023-09-06T18:54:38.469 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: getaddrinfo returned 2 records 2023-09-06T18:54:38.469 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: Attempting to connect using address record (, , 6, '', ('::1', 5672, 0, 0)) 2023-09-06T18:54:38.469 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: INFO: Pika version 1.3.1 connecting to ('::1', 5672, 0, 0) 2023-09-06T18:54:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: add_callback_threadsafe: added callback=> 2023-09-06T18:54:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1108.140905853 and callback=>; now=1098.140905853; delay=10.0 2023-09-06T18:54:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1113.1409359 and callback=>; now=1098.1409359; delay=15.0 2023-09-06T18:54:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: process_timeouts: invoking callback=> 2023-09-06T18:54:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:38.471 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) added handler Wr 2023-09-06T18:54:38.471 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Connection-establishment is in progress for . 2023-09-06T18:54:38.471 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: INFO: Socket connected: 2023-09-06T18:54:38.471 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _AsyncSocketConnector._report_completion(None); 2023-09-06T18:54:38.471 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:38.472 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) removed handler 2023-09-06T18:54:38.472 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: remove_timeout: removing timeout with deadline=1108.140905853 and callback=> 2023-09-06T18:54:38.472 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: TCP connection to broker established: . 2023-09-06T18:54:38.472 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _AsyncStreamConnector.start(); 2023-09-06T18:54:38.472 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: add_callback_threadsafe: added callback=> 2023-09-06T18:54:38.473 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: process_timeouts: invoking callback=> 2023-09-06T18:54:38.473 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _AsyncStreamConnector._start_async(); 2023-09-06T18:54:38.473 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _AsyncStreamConnector._linkup() 2023-09-06T18:54:38.473 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: New Connection state: CLOSED (prev=CLOSED) 2023-09-06T18:54:38.473 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.473 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.474 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >>, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.474 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: New Connection state: INIT (prev=CLOSED) 2023-09-06T18:54:38.474 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: Using external connection workflow. 2023-09-06T18:54:38.474 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _AsyncTransportBase.__init__: 2023-09-06T18:54:38.474 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_reader(15, >) 2023-09-06T18:54:38.475 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_reader(15, _) added handler Rd 2023-09-06T18:54:38.475 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _linkup(): created transport 2023-09-06T18:54:38.475 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: New Connection state: PROTOCOL (prev=INIT) 2023-09-06T18:54:38.475 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:38.475 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:38.476 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.476 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _linkup(): introduced transport to protocol ; _StreamingProtocolShim: params=> 2023-09-06T18:54:38.476 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _AsyncStreamConnector._report_completion((, _StreamingProtocolShim: params=>)); 2023-09-06T18:54:38.476 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: INFO: Streaming transport linked up: (, _StreamingProtocolShim: params=>). 2023-09-06T18:54:38.477 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.477 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.477 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.477 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: _AsyncStreamConnector._cleanup(False) 2023-09-06T18:54:38.477 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:38.478 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:38.478 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.478 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 0:Connection.Start 2023-09-06T18:54:38.478 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.478 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.479 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.479 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>> for "0:Connection.Start" 2023-09-06T18:54:38.479 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: New Connection state: START (prev=PROTOCOL) 2023-09-06T18:54:38.479 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.479 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:38.479 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:38.480 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.480 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.480 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:38.480 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:38.480 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.481 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 0:Connection.Tune 2023-09-06T18:54:38.481 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.481 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.481 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.481 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>> for "0:Connection.Tune" 2023-09-06T18:54:38.481 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: New Connection state: TUNE (prev=START) 2023-09-06T18:54:38.482 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: Creating a HeartbeatChecker: 60 2023-09-06T18:54:38.482 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: timeout: 60.000000 send_interval: 30.000000 check_interval: 65.000000 2023-09-06T18:54:38.482 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1128.143840176 and callback=>; now=1098.143840176; delay=30.0 2023-09-06T18:54:38.482 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1163.143872049 and callback=>; now=1098.143872049; delay=65 2023-09-06T18:54:38.482 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:38.483 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:38.483 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.483 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.483 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.483 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:38.484 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:38.484 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.484 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 0:Connection.OpenOk 2023-09-06T18:54:38.484 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.484 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.485 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.485 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>> for "0:Connection.OpenOk" 2023-09-06T18:54:38.485 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: New Connection state: OPEN (prev=TUNE) 2023-09-06T18:54:38.485 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 0:_on_connection_open_ok 2023-09-06T18:54:38.485 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling > for "0:_on_connection_open_ok" 2023-09-06T18:54:38.485 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: AMQPConnector: AMQP handshake attempt completed; state=3; error=None; 'localhost'/(, , 6, '', ('::1', 5672, 0, 0)) 2023-09-06T18:54:38.486 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: AMQPConnector: AMQP connection established for 'localhost'/(, , 6, '', ('::1', 5672, 0, 0)): params=> 2023-09-06T18:54:38.486 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: INFO: AMQPConnector - reporting success: params=> 2023-09-06T18:54:38.486 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: remove_timeout: removing timeout with deadline=1113.1409359 and callback=> 2023-09-06T18:54:38.486 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: DEBUG: Connection attempt completed with params=> 2023-09-06T18:54:38.487 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.connection_workflow: INFO: AMQPConnectionWorkflow - reporting success: params=> 2023-09-06T18:54:38.487 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.487 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.blocking_connection: INFO: Connection workflow succeeded: params=> 2023-09-06T18:54:38.487 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.487 INFO:teuthology.orchestra.run.smithi099.stderr:pika.connection: DEBUG: Creating channel 1 2023-09-06T18:54:38.488 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>, 'one_shot': True, 'only': params=>>, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.488 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.488 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.488 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.488 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.489 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:38.489 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:38.489 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.489 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Entering blocking state on frame ; acceptable_replies=[] 2023-09-06T18:54:38.489 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding on_synchronous_complete callback 2023-09-06T18:54:38.490 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.490 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding passed-in RPC response callback 2023-09-06T18:54:38.490 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.490 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>>, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.490 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.491 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>>, 'one_shot': False, 'only': params=>>, 'arguments': None} 2023-09-06T18:54:38.491 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': False, 'only': None, 'arguments': None} 2023-09-06T18:54:38.491 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.blocking_connection: INFO: Created channel=1 2023-09-06T18:54:38.491 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:38.491 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:38.491 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.492 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 1:Channel.OpenOk 2023-09-06T18:54:38.492 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.492 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.492 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.492 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.493 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.493 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.493 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>>> for "1:Channel.OpenOk" 2023-09-06T18:54:38.493 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: 0 blocked frames 2023-09-06T18:54:38.493 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>>> for "1:Channel.OpenOk" 2023-09-06T18:54:38.494 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.494 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:38.494 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:38.494 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.494 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Entering blocking state on frame ; acceptable_replies=[] 2023-09-06T18:54:38.495 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding on_synchronous_complete callback 2023-09-06T18:54:38.495 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.495 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding passed-in RPC response callback 2023-09-06T18:54:38.495 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.495 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:38.496 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:38.496 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.496 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 1:Exchange.DeclareOk 2023-09-06T18:54:38.496 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.496 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.496 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.497 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.497 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.497 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': >, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.497 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>>> for "1:Exchange.DeclareOk" 2023-09-06T18:54:38.497 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: 0 blocked frames 2023-09-06T18:54:38.498 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling > for "1:Exchange.DeclareOk" 2023-09-06T18:54:38.498 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.498 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:38.498 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:38.498 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.499 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Entering blocking state on frame ; acceptable_replies=[] 2023-09-06T18:54:38.499 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding on_synchronous_complete callback 2023-09-06T18:54:38.499 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.499 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding passed-in RPC response callback 2023-09-06T18:54:38.499 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.500 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:38.500 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:38.500 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.500 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 1:Queue.DeclareOk 2023-09-06T18:54:38.500 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.501 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.501 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.501 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.501 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.501 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': >, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.502 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>>> for "1:Queue.DeclareOk" 2023-09-06T18:54:38.502 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: 0 blocked frames 2023-09-06T18:54:38.502 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling > for "1:Queue.DeclareOk" 2023-09-06T18:54:38.502 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.503 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:38.503 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:38.503 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.503 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Entering blocking state on frame ; acceptable_replies=[] 2023-09-06T18:54:38.503 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding on_synchronous_complete callback 2023-09-06T18:54:38.504 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.504 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding passed-in RPC response callback 2023-09-06T18:54:38.504 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': >, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 1} 2023-09-06T18:54:38.504 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:38.504 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:38.505 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.505 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 1:Queue.BindOk 2023-09-06T18:54:38.505 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.505 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.505 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.506 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.506 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.506 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #0: {'callback': >, 'one_shot': True, 'only': None, 'arguments': None, 'calls': 0} 2023-09-06T18:54:38.506 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>>> for "1:Queue.BindOk" 2023-09-06T18:54:38.506 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: 0 blocked frames 2023-09-06T18:54:38.507 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling > for "1:Queue.BindOk" 2023-09-06T18:54:38.507 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.507 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:38.507 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:38.507 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:38.508 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Entering blocking state on frame ; acceptable_replies=[(, {'consumer_tag': 'ctag1.43df94ac10f14906a8c54a737ec4ae0e'})] 2023-09-06T18:54:38.508 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding on_synchronous_complete callback 2023-09-06T18:54:38.508 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': {'consumer_tag': 'ctag1.43df94ac10f14906a8c54a737ec4ae0e'}, 'calls': 1} 2023-09-06T18:54:38.508 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Adding passed-in RPC response callback 2023-09-06T18:54:38.508 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Added: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': {'consumer_tag': 'ctag1.43df94ac10f14906a8c54a737ec4ae0e'}, 'calls': 1} 2023-09-06T18:54:38.509 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:38.509 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:38.509 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:38.509 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing 1:Basic.ConsumeOk 2023-09-06T18:54:38.509 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.510 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.510 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Comparing {'consumer_tag': 'ctag1.43df94ac10f14906a8c54a737ec4ae0e'} to {'consumer_tag': 'ctag1.43df94ac10f14906a8c54a737ec4ae0e'} 2023-09-06T18:54:38.510 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #1: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': {'consumer_tag': 'ctag1.43df94ac10f14906a8c54a737ec4ae0e'}, 'calls': 0} 2023-09-06T18:54:38.510 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Processing use of oneshot callback 2023-09-06T18:54:38.511 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: 0 registered uses left 2023-09-06T18:54:38.511 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Comparing {'consumer_tag': 'ctag1.43df94ac10f14906a8c54a737ec4ae0e'} to {'consumer_tag': 'ctag1.43df94ac10f14906a8c54a737ec4ae0e'} 2023-09-06T18:54:38.511 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Removing callback #1: {'callback': params=>>>, 'one_shot': True, 'only': None, 'arguments': {'consumer_tag': 'ctag1.43df94ac10f14906a8c54a737ec4ae0e'}, 'calls': 0} 2023-09-06T18:54:38.511 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling > for "1:Basic.ConsumeOk" 2023-09-06T18:54:38.511 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>>> for "1:Basic.ConsumeOk" 2023-09-06T18:54:38.512 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: 0 blocked frames 2023-09-06T18:54:38.512 INFO:teuthology.orchestra.run.smithi099.stderr:pika.callback: DEBUG: Calling params=>>> for "1:Basic.ConsumeOk" 2023-09-06T18:54:38.512 INFO:teuthology.orchestra.run.smithi099.stderr:pika.channel: DEBUG: Discarding frame "])> 2023-09-06T18:54:38.512 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.513 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP receiver started 2023-09-06T18:54:38.513 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP receiver started 2023-09-06T18:54:38.513 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-service-name: calling handler 2023-09-06T18:54:38.513 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event creating-client-class.sns: calling handler 2023-09-06T18:54:38.513 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.regions: DEBUG: Creating a regex based endpoint for sns, default 2023-09-06T18:54:38.514 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Setting sns timeout as (60, 60) 2023-09-06T18:54:38.514 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.client: DEBUG: Registering retry handlers for service: sns 2023-09-06T18:54:38.514 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-parameter-build.sns.CreateTopic: calling handler 2023-09-06T18:54:38.514 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-call.sns.CreateTopic: calling handler 2023-09-06T18:54:38.514 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-call.sns.CreateTopic: calling handler 2023-09-06T18:54:38.514 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Making request for OperationModel(name=CreateTopic) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.23.10 Python/3.6.8 Linux/4.18.0-513.el8.x86_64 Botocore/1.26.10'}, 'body': {'Action': 'CreateTopic', 'Version': '2010-03-31', 'Name': 'cjhpse-2_topic_1', 'Attributes.entry.1.key': 'push-endpoint', 'Attributes.entry.1.value': 'amqp://localhost', 'Attributes.entry.2.key': 'amqp-exchange', 'Attributes.entry.2.value': 'ex1', 'Attributes.entry.3.key': 'amqp-ack-level', 'Attributes.entry.3.value': 'broker'}, 'url': 'http://smithi099.front.sepia.ceph.com:80/', 'context': {'client_region': 'default', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2023-09-06T18:54:38.515 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.sns.CreateTopic: calling handler > 2023-09-06T18:54:38.515 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.sns.CreateTopic: calling handler 2023-09-06T18:54:38.515 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Calculating signature using v4 auth. 2023-09-06T18:54:38.515 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: CanonicalRequest: 2023-09-06T18:54:38.516 INFO:teuthology.orchestra.run.smithi099.stderr:POST 2023-09-06T18:54:38.516 INFO:teuthology.orchestra.run.smithi099.stderr:/ 2023-09-06T18:54:38.516 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:38.516 INFO:teuthology.orchestra.run.smithi099.stderr:content-type:application/x-www-form-urlencoded; charset=utf-8 2023-09-06T18:54:38.516 INFO:teuthology.orchestra.run.smithi099.stderr:host:smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.516 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-date:20230906T184110Z 2023-09-06T18:54:38.517 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:38.517 INFO:teuthology.orchestra.run.smithi099.stderr:content-type;host;x-amz-date 2023-09-06T18:54:38.517 INFO:teuthology.orchestra.run.smithi099.stderr:96febaebefc4ce86bf0b80a0f548276d5deda4a676accd58190fd45c82d60d98 2023-09-06T18:54:38.517 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: StringToSign: 2023-09-06T18:54:38.517 INFO:teuthology.orchestra.run.smithi099.stderr:AWS4-HMAC-SHA256 2023-09-06T18:54:38.518 INFO:teuthology.orchestra.run.smithi099.stderr:20230906T184110Z 2023-09-06T18:54:38.518 INFO:teuthology.orchestra.run.smithi099.stderr:20230906/default/sns/aws4_request 2023-09-06T18:54:38.518 INFO:teuthology.orchestra.run.smithi099.stderr:11fce26a4fb16d41611b60ff0914f822276cf954d039bbd6f932a0676d73a3f9 2023-09-06T18:54:38.518 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Signature: 2023-09-06T18:54:38.518 INFO:teuthology.orchestra.run.smithi099.stderr:0dc820131307e23557aeb0a295313033923631b8c5449989e2d33a131706f9de 2023-09-06T18:54:38.519 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.sns.CreateTopic: calling handler 2023-09-06T18:54:38.519 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Sending http request: 2023-09-06T18:54:38.519 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): smithi099.front.sepia.ceph.com:80 2023-09-06T18:54:38.519 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.connectionpool: DEBUG: http://smithi099.front.sepia.ceph.com:80 "POST / HTTP/1.1" 200 312 2023-09-06T18:54:38.520 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.parsers: DEBUG: Response headers: {'x-amz-request-id': 'tx00000c8585fd9229b1db7-0064f8c7c6-11b2-default', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (reef)', 'Content-Length': '312', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Connection': 'Keep-Alive'} 2023-09-06T18:54:38.520 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.parsers: DEBUG: Response body: 2023-09-06T18:54:38.520 INFO:teuthology.orchestra.run.smithi099.stderr:b'arn:aws:sns:default::cjhpse-2_topic_1d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.14436393991729257911' 2023-09-06T18:54:38.520 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event needs-retry.sns.CreateTopic: calling handler 2023-09-06T18:54:38.520 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.retryhandler: DEBUG: No retry needed. 2023-09-06T18:54:38.521 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-service-name: calling handler 2023-09-06T18:54:38.521 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event creating-client-class.sns: calling handler 2023-09-06T18:54:38.521 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.regions: DEBUG: Creating a regex based endpoint for sns, default 2023-09-06T18:54:38.521 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Setting sns timeout as (60, 60) 2023-09-06T18:54:38.521 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.client: DEBUG: Registering retry handlers for service: sns 2023-09-06T18:54:38.522 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-parameter-build.sns.CreateTopic: calling handler 2023-09-06T18:54:38.522 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-call.sns.CreateTopic: calling handler 2023-09-06T18:54:38.522 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-call.sns.CreateTopic: calling handler 2023-09-06T18:54:38.522 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Making request for OperationModel(name=CreateTopic) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.23.10 Python/3.6.8 Linux/4.18.0-513.el8.x86_64 Botocore/1.26.10'}, 'body': {'Action': 'CreateTopic', 'Version': '2010-03-31', 'Name': 'cjhpse-2_topic_2', 'Attributes.entry.1.key': 'push-endpoint', 'Attributes.entry.1.value': 'amqp://localhost', 'Attributes.entry.2.key': 'amqp-exchange', 'Attributes.entry.2.value': 'ex1', 'Attributes.entry.3.key': 'amqp-ack-level', 'Attributes.entry.3.value': 'routable'}, 'url': 'http://smithi099.front.sepia.ceph.com:80/', 'context': {'client_region': 'default', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2023-09-06T18:54:38.522 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.sns.CreateTopic: calling handler > 2023-09-06T18:54:38.523 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.sns.CreateTopic: calling handler 2023-09-06T18:54:38.523 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Calculating signature using v4 auth. 2023-09-06T18:54:38.523 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: CanonicalRequest: 2023-09-06T18:54:38.523 INFO:teuthology.orchestra.run.smithi099.stderr:POST 2023-09-06T18:54:38.523 INFO:teuthology.orchestra.run.smithi099.stderr:/ 2023-09-06T18:54:38.523 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:content-type:application/x-www-form-urlencoded; charset=utf-8 2023-09-06T18:54:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:host:smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-date:20230906T184110Z 2023-09-06T18:54:38.524 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:content-type;host;x-amz-date 2023-09-06T18:54:38.525 INFO:teuthology.orchestra.run.smithi099.stderr:90eb1216bf8b1b14a7714f69ba3ea7a88b416e4f447378fa0a8b31f367af67d9 2023-09-06T18:54:38.525 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: StringToSign: 2023-09-06T18:54:38.525 INFO:teuthology.orchestra.run.smithi099.stderr:AWS4-HMAC-SHA256 2023-09-06T18:54:38.525 INFO:teuthology.orchestra.run.smithi099.stderr:20230906T184110Z 2023-09-06T18:54:38.525 INFO:teuthology.orchestra.run.smithi099.stderr:20230906/default/sns/aws4_request 2023-09-06T18:54:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:448b885d6c421483d9ced64f1fd823e51669a3f5a26474dd8a7465e9ff7224d2 2023-09-06T18:54:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Signature: 2023-09-06T18:54:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:ff9fbc983ff09eadf8f3f6b823d339386573432340eeca0c0c828e80417c1870 2023-09-06T18:54:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.sns.CreateTopic: calling handler 2023-09-06T18:54:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Sending http request: 2023-09-06T18:54:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): smithi099.front.sepia.ceph.com:80 2023-09-06T18:54:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.connectionpool: DEBUG: http://smithi099.front.sepia.ceph.com:80 "POST / HTTP/1.1" 200 311 2023-09-06T18:54:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.parsers: DEBUG: Response headers: {'x-amz-request-id': 'tx00000832c4565925151ee-0064f8c7c6-11b2-default', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (reef)', 'Content-Length': '311', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Connection': 'Keep-Alive'} 2023-09-06T18:54:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.parsers: DEBUG: Response body: 2023-09-06T18:54:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:b'arn:aws:sns:default::cjhpse-2_topic_2d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9452006020492644846' 2023-09-06T18:54:38.528 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event needs-retry.sns.CreateTopic: calling handler 2023-09-06T18:54:38.528 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.retryhandler: DEBUG: No retry needed. 2023-09-06T18:54:38.528 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-service-name: calling handler 2023-09-06T18:54:38.528 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler 2023-09-06T18:54:38.528 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler ._handler at 0x7f5e9314f378> 2023-09-06T18:54:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler 2023-09-06T18:54:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Setting s3 timeout as (60, 60) 2023-09-06T18:54:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.client: DEBUG: Registering retry handlers for service: s3 2023-09-06T18:54:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler 2023-09-06T18:54:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler > 2023-09-06T18:54:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler > 2023-09-06T18:54:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler 2023-09-06T18:54:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2023-09-06T18:54:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-call.s3.PutBucketNotificationConfiguration: calling handler > 2023-09-06T18:54:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2023-09-06T18:54:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2023-09-06T18:54:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Making request for OperationModel(name=PutBucketNotificationConfiguration) with params: {'url_path': '/cjhpse-2?notification', 'query_string': {}, 'method': 'PUT', 'headers': {'User-Agent': 'Boto3/1.23.10 Python/3.6.8 Linux/4.18.0-513.el8.x86_64 Botocore/1.26.10'}, 'body': b'cjhpse-2_notif_1arn:aws:sns:default::cjhpse-2_topic_1cjhpse-2_notif_2arn:aws:sns:default::cjhpse-2_topic_2s3:ObjectCreated:*', 'url': 'http://smithi099.front.sepia.ceph.com:80/cjhpse-2?notification', 'context': {'client_region': 'us-east-1', 'client_config': , 'has_streaming_input': False, 'auth_type': None, 'signing': {'bucket': 'cjhpse-2'}}} 2023-09-06T18:54:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.s3.PutBucketNotificationConfiguration: calling handler > 2023-09-06T18:54:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.PutBucketNotificationConfiguration: calling handler > 2023-09-06T18:54:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.PutBucketNotificationConfiguration: calling handler > 2023-09-06T18:54:38.532 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.PutBucketNotificationConfiguration: calling handler 2023-09-06T18:54:38.532 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-sign.s3.PutBucketNotificationConfiguration: calling handler > 2023-09-06T18:54:38.532 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.utils: DEBUG: Using S3 path style addressing. 2023-09-06T18:54:38.532 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Calculating signature using v4 auth. 2023-09-06T18:54:38.532 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: CanonicalRequest: 2023-09-06T18:54:38.533 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.533 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2 2023-09-06T18:54:38.533 INFO:teuthology.orchestra.run.smithi099.stderr:notification= 2023-09-06T18:54:38.533 INFO:teuthology.orchestra.run.smithi099.stderr:host:smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.533 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-content-sha256:8fe47d99835ef5d9add6fd01625cf8053611b6ae603c26d68d69e7b1503637b7 2023-09-06T18:54:38.533 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-date:20230906T184110Z 2023-09-06T18:54:38.534 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:38.534 INFO:teuthology.orchestra.run.smithi099.stderr:host;x-amz-content-sha256;x-amz-date 2023-09-06T18:54:38.534 INFO:teuthology.orchestra.run.smithi099.stderr:8fe47d99835ef5d9add6fd01625cf8053611b6ae603c26d68d69e7b1503637b7 2023-09-06T18:54:38.534 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: StringToSign: 2023-09-06T18:54:38.534 INFO:teuthology.orchestra.run.smithi099.stderr:AWS4-HMAC-SHA256 2023-09-06T18:54:38.535 INFO:teuthology.orchestra.run.smithi099.stderr:20230906T184110Z 2023-09-06T18:54:38.535 INFO:teuthology.orchestra.run.smithi099.stderr:20230906/us-east-1/s3/aws4_request 2023-09-06T18:54:38.535 INFO:teuthology.orchestra.run.smithi099.stderr:414e74dc23ecb1382b558135f98260c59b617c6c6777d811bda0144da4359d99 2023-09-06T18:54:38.535 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Signature: 2023-09-06T18:54:38.535 INFO:teuthology.orchestra.run.smithi099.stderr:5e75670b54073426df762fc5fddb6966e9a87d0942daa234978fc7336e69a9ed 2023-09-06T18:54:38.536 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.s3.PutBucketNotificationConfiguration: calling handler 2023-09-06T18:54:38.536 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Sending http request: 2023-09-06T18:54:38.536 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): smithi099.front.sepia.ceph.com:80 2023-09-06T18:54:38.536 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.connectionpool: DEBUG: http://smithi099.front.sepia.ceph.com:80 "PUT /cjhpse-2?notification HTTP/1.1" 200 0 2023-09-06T18:54:38.537 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.parsers: DEBUG: Response headers: {'x-amz-request-id': 'tx00000f1633abd8dc1bd03-0064f8c7c6-11b2-default', 'Server': 'Ceph Object Gateway (reef)', 'Content-Length': '0', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Connection': 'Keep-Alive'} 2023-09-06T18:54:38.537 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.parsers: DEBUG: Response body: 2023-09-06T18:54:38.537 INFO:teuthology.orchestra.run.smithi099.stderr:b'' 2023-09-06T18:54:38.537 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event needs-retry.s3.PutBucketNotificationConfiguration: calling handler 2023-09-06T18:54:38.537 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.retryhandler: DEBUG: No retry needed. 2023-09-06T18:54:38.538 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event needs-retry.s3.PutBucketNotificationConfiguration: calling handler > 2023-09-06T18:54:38.538 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/0 2023-09-06T18:54:38.538 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/0 2023-09-06T18:54:38.538 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.538 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/0 2023-09-06T18:54:38.539 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.539 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '/2yftLj5fD1cV+S+8TXNhQ==', 'Content-Length': '3009790', 'Expect': '100-Continue'} 2023-09-06T18:54:38.539 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.539 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.539 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.540 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.540 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.540 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.540 INFO:teuthology.orchestra.run.smithi099.stderr:/2yftLj5fD1cV+S+8TXNhQ== 2023-09-06T18:54:38.540 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.540 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:38.541 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/0 2023-09-06T18:54:38.541 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.541 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:oxj86sCXqrZj2jJsJJ6oqpOoQHo= 2023-09-06T18:54:38.541 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '/2yftLj5fD1cV+S+8TXNhQ==', 'Content-Length': '3009790', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:oxj86sCXqrZj2jJsJJ6oqpOoQHo='} 2023-09-06T18:54:38.541 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/1 2023-09-06T18:54:38.542 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/1 2023-09-06T18:54:38.542 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.542 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/1 2023-09-06T18:54:38.542 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.542 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'VPGZaGuDeXSY+pMirkHpUg==', 'Content-Length': '3009350', 'Expect': '100-Continue'} 2023-09-06T18:54:38.543 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.543 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.543 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.543 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.543 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.543 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.544 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.544 INFO:teuthology.orchestra.run.smithi099.stderr:VPGZaGuDeXSY+pMirkHpUg== 2023-09-06T18:54:38.544 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.544 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:38.544 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/1 2023-09-06T18:54:38.545 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.545 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:M4ICs2HobSbZ6esIYvgMbay1Yjk= 2023-09-06T18:54:38.545 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'VPGZaGuDeXSY+pMirkHpUg==', 'Content-Length': '3009350', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:M4ICs2HobSbZ6esIYvgMbay1Yjk='} 2023-09-06T18:54:38.545 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/2 2023-09-06T18:54:38.545 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/2 2023-09-06T18:54:38.545 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.546 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/2 2023-09-06T18:54:38.546 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.546 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ziFK9FW+tI4Ckxbr6zYgsA==', 'Content-Length': '3008326', 'Expect': '100-Continue'} 2023-09-06T18:54:38.546 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.546 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.547 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.547 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.547 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.547 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.548 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.548 INFO:teuthology.orchestra.run.smithi099.stderr:ziFK9FW+tI4Ckxbr6zYgsA== 2023-09-06T18:54:38.548 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.548 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:38.548 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/2 2023-09-06T18:54:38.549 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.549 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Vy2A3FDf7tKT02X1zzqcAwn2y6Q= 2023-09-06T18:54:38.549 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ziFK9FW+tI4Ckxbr6zYgsA==', 'Content-Length': '3008326', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Vy2A3FDf7tKT02X1zzqcAwn2y6Q='} 2023-09-06T18:54:38.549 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/3 2023-09-06T18:54:38.549 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/3 2023-09-06T18:54:38.550 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.550 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/3 2023-09-06T18:54:38.550 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.550 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rcjRGgFvlYdygS7hxP0+hA==', 'Content-Length': '3011691', 'Expect': '100-Continue'} 2023-09-06T18:54:38.550 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.551 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.551 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.551 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.551 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.551 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.552 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.552 INFO:teuthology.orchestra.run.smithi099.stderr:rcjRGgFvlYdygS7hxP0+hA== 2023-09-06T18:54:38.552 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.552 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:38.552 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/3 2023-09-06T18:54:38.553 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.553 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:d19aDB3+lcy6ioJ4o+44D2tLCT8= 2023-09-06T18:54:38.553 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rcjRGgFvlYdygS7hxP0+hA==', 'Content-Length': '3011691', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:d19aDB3+lcy6ioJ4o+44D2tLCT8='} 2023-09-06T18:54:38.553 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.553 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.553 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.783006Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.17693029742920458362","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"0","size":3009790,"eTag":"ff6c9fb4b8f97c3d5c57e4bef135cd85","versionId":"","sequencer":"C6C7F864E5F4A92F","metadata":[],"tags":[]}},"eventId":"1694025670.799667.ff6c9fb4b8f97c3d5c57e4bef135cd85","opaqueData":""}]}' 2023-09-06T18:54:38.554 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.554 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.798927Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.8481461665646232376","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"1","size":3009350,"eTag":"54f199686b83797498fa9322ae41e952","versionId":"","sequencer":"C6C7F8645F716730","metadata":[],"tags":[]}},"eventId":"1694025670.812085.54f199686b83797498fa9322ae41e952","opaqueData":""}]}' 2023-09-06T18:54:38.554 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/4 2023-09-06T18:54:38.554 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/4 2023-09-06T18:54:38.554 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.555 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/4 2023-09-06T18:54:38.555 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.555 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'A/MLHQ6IsilyH/yB8CvJDg==', 'Content-Length': '3014736', 'Expect': '100-Continue'} 2023-09-06T18:54:38.555 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.555 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.555 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.556 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.556 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.556 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.556 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.557 INFO:teuthology.orchestra.run.smithi099.stderr:A/MLHQ6IsilyH/yB8CvJDg== 2023-09-06T18:54:38.557 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.557 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:38.557 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/4 2023-09-06T18:54:38.557 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.558 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:lAH2tuI2pYH68nzPsug+3gbi1ms= 2023-09-06T18:54:38.558 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'A/MLHQ6IsilyH/yB8CvJDg==', 'Content-Length': '3014736', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:lAH2tuI2pYH68nzPsug+3gbi1ms='} 2023-09-06T18:54:38.558 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/5 2023-09-06T18:54:38.558 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/5 2023-09-06T18:54:38.558 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.559 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/5 2023-09-06T18:54:38.559 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.559 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'VrLdFn/AOR/U0R7Gu0o3Yw==', 'Content-Length': '3014004', 'Expect': '100-Continue'} 2023-09-06T18:54:38.559 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.559 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.559 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.560 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.560 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.560 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.560 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.560 INFO:teuthology.orchestra.run.smithi099.stderr:VrLdFn/AOR/U0R7Gu0o3Yw== 2023-09-06T18:54:38.561 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.561 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:38.561 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/5 2023-09-06T18:54:38.561 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.561 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:2be3TXrp9/Lulr0Gm7p2M8tot3k= 2023-09-06T18:54:38.561 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'VrLdFn/AOR/U0R7Gu0o3Yw==', 'Content-Length': '3014004', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:2be3TXrp9/Lulr0Gm7p2M8tot3k='} 2023-09-06T18:54:38.562 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/6 2023-09-06T18:54:38.562 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/6 2023-09-06T18:54:38.562 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.562 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/6 2023-09-06T18:54:38.562 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.563 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'CIlyOC6kL5DZh/KzQ5Q5Dg==', 'Content-Length': '3008365', 'Expect': '100-Continue'} 2023-09-06T18:54:38.563 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.563 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.563 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.563 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:CIlyOC6kL5DZh/KzQ5Q5Dg== 2023-09-06T18:54:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:38.565 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/6 2023-09-06T18:54:38.565 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.565 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:p0rBW8dlQWkbV+iI58Wk3zfHFAY= 2023-09-06T18:54:38.565 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'CIlyOC6kL5DZh/KzQ5Q5Dg==', 'Content-Length': '3008365', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:p0rBW8dlQWkbV+iI58Wk3zfHFAY='} 2023-09-06T18:54:38.565 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/7 2023-09-06T18:54:38.566 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/7 2023-09-06T18:54:38.566 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.566 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/7 2023-09-06T18:54:38.566 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.566 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'tHMq8RBR/3xmeI5JNrrmkA==', 'Content-Length': '3011758', 'Expect': '100-Continue'} 2023-09-06T18:54:38.566 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.567 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.567 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.567 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.567 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.568 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.568 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.568 INFO:teuthology.orchestra.run.smithi099.stderr:tHMq8RBR/3xmeI5JNrrmkA== 2023-09-06T18:54:38.568 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.568 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:38.569 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/7 2023-09-06T18:54:38.569 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.569 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:dTRC72JpWd3UG7VU1VG56EHpock= 2023-09-06T18:54:38.569 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'tHMq8RBR/3xmeI5JNrrmkA==', 'Content-Length': '3011758', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:dTRC72JpWd3UG7VU1VG56EHpock='} 2023-09-06T18:54:38.569 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/8 2023-09-06T18:54:38.570 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/8 2023-09-06T18:54:38.570 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.570 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/8 2023-09-06T18:54:38.570 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.570 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '256+Zv3/4kMG5Afi+veJqw==', 'Content-Length': '3010227', 'Expect': '100-Continue'} 2023-09-06T18:54:38.570 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.571 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.571 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.571 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.571 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.572 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.572 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.572 INFO:teuthology.orchestra.run.smithi099.stderr:256+Zv3/4kMG5Afi+veJqw== 2023-09-06T18:54:38.572 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.572 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:38.573 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/8 2023-09-06T18:54:38.573 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.573 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:fs4qDwXDMtwpmODAEUFwUzkBUe8= 2023-09-06T18:54:38.573 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '256+Zv3/4kMG5Afi+veJqw==', 'Content-Length': '3010227', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:fs4qDwXDMtwpmODAEUFwUzkBUe8='} 2023-09-06T18:54:38.573 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.574 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.574 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.812854Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.13742554559163728075","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"2","size":3008326,"eTag":"ce214af455beb48e029316ebeb3620b0","versionId":"","sequencer":"C6C7F864B2786B31","metadata":[],"tags":[]}},"eventId":"1694025670.829126.ce214af455beb48e029316ebeb3620b0","opaqueData":""}]}' 2023-09-06T18:54:38.574 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.574 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.844276Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9602259965651842522","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"3","size":3011691,"eTag":"adc8d11a016f958772812ee1c4fd3e84","versionId":"","sequencer":"C6C7F86430D61C33","metadata":[],"tags":[]}},"eventId":"1694025670.857527.adc8d11a016f958772812ee1c4fd3e84","opaqueData":""}]}' 2023-09-06T18:54:38.574 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.575 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.871425Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9177669767589033870","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"4","size":3014736,"eTag":"03f30b1d0e88b229721ffc81f02bc90e","versionId":"","sequencer":"C6C7F8644860A234","metadata":[],"tags":[]}},"eventId":"1694025670.883056.03f30b1d0e88b229721ffc81f02bc90e","opaqueData":""}]}' 2023-09-06T18:54:38.575 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/9 2023-09-06T18:54:38.575 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/9 2023-09-06T18:54:38.575 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.575 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/9 2023-09-06T18:54:38.576 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.576 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TifjY0DRwhZKD2/uITKxzA==', 'Content-Length': '3008592', 'Expect': '100-Continue'} 2023-09-06T18:54:38.576 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.576 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.576 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:TifjY0DRwhZKD2/uITKxzA== 2023-09-06T18:54:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.578 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:38.578 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/9 2023-09-06T18:54:38.578 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.578 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:ZQX0BpYO8QA8f9YvNmQGKegAn40= 2023-09-06T18:54:38.578 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TifjY0DRwhZKD2/uITKxzA==', 'Content-Length': '3008592', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:ZQX0BpYO8QA8f9YvNmQGKegAn40='} 2023-09-06T18:54:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/10 2023-09-06T18:54:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/10 2023-09-06T18:54:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/10 2023-09-06T18:54:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'XHQwrFyC/jDkrxHQHU8heQ==', 'Content-Length': '3010335', 'Expect': '100-Continue'} 2023-09-06T18:54:38.580 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.580 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.580 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.580 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.580 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.581 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.581 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.581 INFO:teuthology.orchestra.run.smithi099.stderr:XHQwrFyC/jDkrxHQHU8heQ== 2023-09-06T18:54:38.581 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.581 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:38.582 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/10 2023-09-06T18:54:38.582 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.582 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:nNHvf+3oYzI6qi4psurZAW6xnhA= 2023-09-06T18:54:38.582 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'XHQwrFyC/jDkrxHQHU8heQ==', 'Content-Length': '3010335', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:nNHvf+3oYzI6qi4psurZAW6xnhA='} 2023-09-06T18:54:38.582 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/11 2023-09-06T18:54:38.583 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/11 2023-09-06T18:54:38.583 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.583 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/11 2023-09-06T18:54:38.583 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.583 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'AnMovVCVamDY4RSD/lNBOQ==', 'Content-Length': '3011165', 'Expect': '100-Continue'} 2023-09-06T18:54:38.584 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.584 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.584 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.584 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.584 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.584 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.585 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.585 INFO:teuthology.orchestra.run.smithi099.stderr:AnMovVCVamDY4RSD/lNBOQ== 2023-09-06T18:54:38.585 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.585 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:38.585 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/11 2023-09-06T18:54:38.585 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.586 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:xAoeK4NuACdMKu8XsC5Z5pKX5z4= 2023-09-06T18:54:38.586 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'AnMovVCVamDY4RSD/lNBOQ==', 'Content-Length': '3011165', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:10 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:xAoeK4NuACdMKu8XsC5Z5pKX5z4='} 2023-09-06T18:54:38.586 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/12 2023-09-06T18:54:38.586 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/12 2023-09-06T18:54:38.587 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.587 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/12 2023-09-06T18:54:38.587 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.587 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3BZQx1dDfTZgv/igiRmnLQ==', 'Content-Length': '3010730', 'Expect': '100-Continue'} 2023-09-06T18:54:38.587 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.588 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.588 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.588 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.588 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.588 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.588 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.589 INFO:teuthology.orchestra.run.smithi099.stderr:3BZQx1dDfTZgv/igiRmnLQ== 2023-09-06T18:54:38.589 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.589 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.589 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/12 2023-09-06T18:54:38.589 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:x1MsA3boDTeCday2NNxv/cRNrsQ= 2023-09-06T18:54:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3BZQx1dDfTZgv/igiRmnLQ==', 'Content-Length': '3010730', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:x1MsA3boDTeCday2NNxv/cRNrsQ='} 2023-09-06T18:54:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.591 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.783006Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.17693029742920458362","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"0","size":3009790,"eTag":"ff6c9fb4b8f97c3d5c57e4bef135cd85","versionId":"","sequencer":"C6C7F8640AFC6D36","metadata":[],"tags":[]}},"eventId":"1694025670.913177.ff6c9fb4b8f97c3d5c57e4bef135cd85","opaqueData":""}]}' 2023-09-06T18:54:38.591 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.591 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.798927Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.8481461665646232376","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"1","size":3009350,"eTag":"54f199686b83797498fa9322ae41e952","versionId":"","sequencer":"C6C7F8647DFB6D36","metadata":[],"tags":[]}},"eventId":"1694025670.913177.54f199686b83797498fa9322ae41e952","opaqueData":""}]}' 2023-09-06T18:54:38.591 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.591 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.592 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.902271Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.4250564861246872994","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"5","size":3014004,"eTag":"56b2dd167fc0391fd4d11ec6bb4a3763","versionId":"","sequencer":"C6C7F86478588936","metadata":[],"tags":[]}},"eventId":"1694025670.914970.56b2dd167fc0391fd4d11ec6bb4a3763","opaqueData":""}]}' 2023-09-06T18:54:38.592 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.592 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.943327Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.17355266024743407755","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"6","size":3008365,"eTag":"088972382ea42f90d987f2b34394390e","versionId":"","sequencer":"C6C7F86404F93339","metadata":[],"tags":[]}},"eventId":"1694025670.959707.088972382ea42f90d987f2b34394390e","opaqueData":""}]}' 2023-09-06T18:54:38.592 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.592 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.977928Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.10768406405656094381","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"7","size":3011758,"eTag":"b4732af11051ff7c66788e4936bae690","versionId":"","sequencer":"C6C7F864FE22073B","metadata":[],"tags":[]}},"eventId":"1694025670.990323.b4732af11051ff7c66788e4936bae690","opaqueData":""}]}' 2023-09-06T18:54:38.593 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/13 2023-09-06T18:54:38.593 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/13 2023-09-06T18:54:38.593 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.593 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/13 2023-09-06T18:54:38.593 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.593 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '7WlJaOkOlGmUWVaO4TE4xw==', 'Content-Length': '3007999', 'Expect': '100-Continue'} 2023-09-06T18:54:38.594 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.594 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.594 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.594 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.594 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.595 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.595 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.595 INFO:teuthology.orchestra.run.smithi099.stderr:7WlJaOkOlGmUWVaO4TE4xw== 2023-09-06T18:54:38.595 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.595 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.596 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/13 2023-09-06T18:54:38.596 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.596 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:wHscuwAyCf0cC+3xKCWxF7tlEmM= 2023-09-06T18:54:38.596 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '7WlJaOkOlGmUWVaO4TE4xw==', 'Content-Length': '3007999', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:wHscuwAyCf0cC+3xKCWxF7tlEmM='} 2023-09-06T18:54:38.596 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/14 2023-09-06T18:54:38.596 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/14 2023-09-06T18:54:38.597 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.597 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/14 2023-09-06T18:54:38.597 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.597 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'zaRIXJyU53biENFoirVQ0A==', 'Content-Length': '3010514', 'Expect': '100-Continue'} 2023-09-06T18:54:38.597 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.597 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.598 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.598 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.598 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.598 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.599 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.599 INFO:teuthology.orchestra.run.smithi099.stderr:zaRIXJyU53biENFoirVQ0A== 2023-09-06T18:54:38.599 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.599 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.599 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/14 2023-09-06T18:54:38.599 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.600 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:g54c93X3MbIM7oz66LzKLp9KocY= 2023-09-06T18:54:38.600 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'zaRIXJyU53biENFoirVQ0A==', 'Content-Length': '3010514', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:g54c93X3MbIM7oz66LzKLp9KocY='} 2023-09-06T18:54:38.600 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/15 2023-09-06T18:54:38.600 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/15 2023-09-06T18:54:38.600 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.601 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/15 2023-09-06T18:54:38.601 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.601 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1vVrfom5Cd3jU937DQb/+Q==', 'Content-Length': '3011089', 'Expect': '100-Continue'} 2023-09-06T18:54:38.601 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.602 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.602 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.602 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.602 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.602 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.603 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.603 INFO:teuthology.orchestra.run.smithi099.stderr:1vVrfom5Cd3jU937DQb/+Q== 2023-09-06T18:54:38.603 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.603 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.603 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/15 2023-09-06T18:54:38.603 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.604 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:SMZkhiEn9TNiejn1ycMIfIwviAk= 2023-09-06T18:54:38.604 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1vVrfom5Cd3jU937DQb/+Q==', 'Content-Length': '3011089', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:SMZkhiEn9TNiejn1ycMIfIwviAk='} 2023-09-06T18:54:38.604 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/16 2023-09-06T18:54:38.604 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/16 2023-09-06T18:54:38.604 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.605 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/16 2023-09-06T18:54:38.605 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.605 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'SvrtkFR58CQWPSbWl5Me2A==', 'Content-Length': '3009990', 'Expect': '100-Continue'} 2023-09-06T18:54:38.605 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.605 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.605 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.606 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.606 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.606 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.606 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.606 INFO:teuthology.orchestra.run.smithi099.stderr:SvrtkFR58CQWPSbWl5Me2A== 2023-09-06T18:54:38.607 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.607 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.607 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/16 2023-09-06T18:54:38.607 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.607 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:pi5Xoo/rjLBL84qRbVCFPE6qqL8= 2023-09-06T18:54:38.607 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'SvrtkFR58CQWPSbWl5Me2A==', 'Content-Length': '3009990', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:pi5Xoo/rjLBL84qRbVCFPE6qqL8='} 2023-09-06T18:54:38.608 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/17 2023-09-06T18:54:38.608 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/17 2023-09-06T18:54:38.608 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.608 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/17 2023-09-06T18:54:38.608 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.608 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'M6MFlxIzG6ArBUqbvB8pQw==', 'Content-Length': '3009758', 'Expect': '100-Continue'} 2023-09-06T18:54:38.609 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.609 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.609 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.609 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.610 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.610 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.610 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.610 INFO:teuthology.orchestra.run.smithi099.stderr:M6MFlxIzG6ArBUqbvB8pQw== 2023-09-06T18:54:38.610 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.611 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.611 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/17 2023-09-06T18:54:38.611 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.611 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:eUksEepfvlkuMLqmNObRHy6cUyQ= 2023-09-06T18:54:38.611 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'M6MFlxIzG6ArBUqbvB8pQw==', 'Content-Length': '3009758', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:eUksEepfvlkuMLqmNObRHy6cUyQ='} 2023-09-06T18:54:38.611 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.612 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.612 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"54f199686b83797498fa9322ae41e952"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000075b43472bcefc338-0064f8c7c6-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.612 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.612 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.844276Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9602259965651842522","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"3","size":3011691,"eTag":"adc8d11a016f958772812ee1c4fd3e84","versionId":"","sequencer":"C7C7F864DC3BCB00","metadata":[],"tags":[]}},"eventId":"1694025671.013319.adc8d11a016f958772812ee1c4fd3e84","opaqueData":""}]}' 2023-09-06T18:54:38.612 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"ff6c9fb4b8f97c3d5c57e4bef135cd85"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f58a44cf3b31e07a-0064f8c7c6-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.613 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.613 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.032044Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.6401212051087397035","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"8","size":3010227,"eTag":"db9ebe66fdffe24306e407e2faf789ab","versionId":"","sequencer":"C7C7F8649A3BB502","metadata":[],"tags":[]}},"eventId":"1694025671.045431.db9ebe66fdffe24306e407e2faf789ab","opaqueData":""}]}' 2023-09-06T18:54:38.613 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.613 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.871425Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9177669767589033870","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"4","size":3014736,"eTag":"03f30b1d0e88b229721ffc81f02bc90e","versionId":"","sequencer":"C7C7F864F53CCB00","metadata":[],"tags":[]}},"eventId":"1694025671.013319.03f30b1d0e88b229721ffc81f02bc90e","opaqueData":""}]}' 2023-09-06T18:54:38.613 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.614 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.065031Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.3982984264661256794","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"9","size":3008592,"eTag":"4e27e36340d1c2164a0f6fee2132b1cc","versionId":"","sequencer":"C7C7F864C349B604","metadata":[],"tags":[]}},"eventId":"1694025671.079055.4e27e36340d1c2164a0f6fee2132b1cc","opaqueData":""}]}' 2023-09-06T18:54:38.614 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.614 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.812854Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.13742554559163728075","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"2","size":3008326,"eTag":"ce214af455beb48e029316ebeb3620b0","versionId":"","sequencer":"C7C7F8641949CB00","metadata":[],"tags":[]}},"eventId":"1694025671.013322.ce214af455beb48e029316ebeb3620b0","opaqueData":""}]}' 2023-09-06T18:54:38.614 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.614 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.097796Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.1071699808165623007","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"10","size":3010335,"eTag":"5c7430ac5c82fe30e4af11d01d4f2179","versionId":"","sequencer":"C7C7F864E42A6706","metadata":[],"tags":[]}},"eventId":"1694025671.107424.5c7430ac5c82fe30e4af11d01d4f2179","opaqueData":""}]}' 2023-09-06T18:54:38.615 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/18 2023-09-06T18:54:38.615 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/18 2023-09-06T18:54:38.615 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.615 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/18 2023-09-06T18:54:38.615 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.615 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lyy/85VIJwToOnkcbGVV7w==', 'Content-Length': '3010849', 'Expect': '100-Continue'} 2023-09-06T18:54:38.616 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.616 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.616 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.616 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.616 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.616 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.617 INFO:teuthology.orchestra.run.smithi099.stderr:Lyy/85VIJwToOnkcbGVV7w== 2023-09-06T18:54:38.617 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.617 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.617 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/18 2023-09-06T18:54:38.618 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.618 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:IVBu0gYngqoEbts/d1WtFWfLq3I= 2023-09-06T18:54:38.618 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lyy/85VIJwToOnkcbGVV7w==', 'Content-Length': '3010849', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:IVBu0gYngqoEbts/d1WtFWfLq3I='} 2023-09-06T18:54:38.618 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/19 2023-09-06T18:54:38.618 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/19 2023-09-06T18:54:38.619 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.619 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/19 2023-09-06T18:54:38.619 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.619 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'bggnuqsyHE6gWj8mIFM+bA==', 'Content-Length': '3012251', 'Expect': '100-Continue'} 2023-09-06T18:54:38.619 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.619 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.620 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.620 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.620 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.620 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.620 INFO:teuthology.orchestra.run.smithi099.stderr:bggnuqsyHE6gWj8mIFM+bA== 2023-09-06T18:54:38.621 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.621 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.621 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/19 2023-09-06T18:54:38.621 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.621 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:zM/1t1Mx2R5gG4/mmvZ1crFBtPE= 2023-09-06T18:54:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'bggnuqsyHE6gWj8mIFM+bA==', 'Content-Length': '3012251', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:zM/1t1Mx2R5gG4/mmvZ1crFBtPE='} 2023-09-06T18:54:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/20 2023-09-06T18:54:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/20 2023-09-06T18:54:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/20 2023-09-06T18:54:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.623 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'LrG3j4U3mBaQJNLJfSxqIw==', 'Content-Length': '3010394', 'Expect': '100-Continue'} 2023-09-06T18:54:38.623 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.623 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.623 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.623 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:LrG3j4U3mBaQJNLJfSxqIw== 2023-09-06T18:54:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.625 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.625 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/20 2023-09-06T18:54:38.625 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.625 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:IWntYpkrn0IKu8W6sZIme+uqo88= 2023-09-06T18:54:38.625 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'LrG3j4U3mBaQJNLJfSxqIw==', 'Content-Length': '3010394', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:IWntYpkrn0IKu8W6sZIme+uqo88='} 2023-09-06T18:54:38.626 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/21 2023-09-06T18:54:38.626 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/21 2023-09-06T18:54:38.626 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.626 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/21 2023-09-06T18:54:38.626 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.627 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'djXv/DZfdgAXwqtGTD4O4w==', 'Content-Length': '3009927', 'Expect': '100-Continue'} 2023-09-06T18:54:38.627 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.627 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.627 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.627 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.628 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.628 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.628 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.628 INFO:teuthology.orchestra.run.smithi099.stderr:djXv/DZfdgAXwqtGTD4O4w== 2023-09-06T18:54:38.628 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.628 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.629 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/21 2023-09-06T18:54:38.629 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.629 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:xNS3u6VFVxdCcGMYcDKy137VYnE= 2023-09-06T18:54:38.629 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'djXv/DZfdgAXwqtGTD4O4w==', 'Content-Length': '3009927', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:xNS3u6VFVxdCcGMYcDKy137VYnE='} 2023-09-06T18:54:38.629 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/22 2023-09-06T18:54:38.630 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/22 2023-09-06T18:54:38.630 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.630 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/22 2023-09-06T18:54:38.630 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.631 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'oR1Byq8sQHgNO1xFsa0kkA==', 'Content-Length': '3010756', 'Expect': '100-Continue'} 2023-09-06T18:54:38.631 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.631 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.631 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.631 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.632 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.632 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.632 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.632 INFO:teuthology.orchestra.run.smithi099.stderr:oR1Byq8sQHgNO1xFsa0kkA== 2023-09-06T18:54:38.632 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.632 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.633 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/22 2023-09-06T18:54:38.633 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.633 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:zhITLmn/JfbddwjtEAqGVFjjv7M= 2023-09-06T18:54:38.633 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'oR1Byq8sQHgNO1xFsa0kkA==', 'Content-Length': '3010756', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:zhITLmn/JfbddwjtEAqGVFjjv7M='} 2023-09-06T18:54:38.633 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"adc8d11a016f958772812ee1c4fd3e84"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008542149055a1f5da-0064f8c7c6-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.633 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"ce214af455beb48e029316ebeb3620b0"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000beb75c8f756100cb-0064f8c7c6-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.634 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.634 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.634 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.142108Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.2757882580617200765","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"11","size":3011165,"eTag":"027328bd50956a60d8e11483fe534139","versionId":"","sequencer":"C7C7F86417C40B09","metadata":[],"tags":[]}},"eventId":"1694025671.151766.027328bd50956a60d8e11483fe534139","opaqueData":""}]}' 2023-09-06T18:54:38.634 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.634 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.159274Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11696590955431901269","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"12","size":3010730,"eTag":"dc1650c757437d3660bff8a08919a72d","versionId":"","sequencer":"C7C7F864F2F7470A","metadata":[],"tags":[]}},"eventId":"1694025671.172488.dc1650c757437d3660bff8a08919a72d","opaqueData":""}]}' 2023-09-06T18:54:38.635 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.635 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.189808Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9394318104517758579","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"13","size":3007999,"eTag":"ed694968e90e94699459568ee13138c7","versionId":"","sequencer":"C7C7F864D39F100C","metadata":[],"tags":[]}},"eventId":"1694025671.202416.ed694968e90e94699459568ee13138c7","opaqueData":""}]}' 2023-09-06T18:54:38.635 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"03f30b1d0e88b229721ffc81f02bc90e"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007f5da2073afc778e-0064f8c7c6-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.635 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.635 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.635 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.977928Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.10768406405656094381","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"7","size":3011758,"eTag":"b4732af11051ff7c66788e4936bae690","versionId":"","sequencer":"C7C7F86497C1C306","metadata":[],"tags":[]}},"eventId":"1694025671.113492.b4732af11051ff7c66788e4936bae690","opaqueData":""}]}' 2023-09-06T18:54:38.636 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.636 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.902271Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.4250564861246872994","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"5","size":3014004,"eTag":"56b2dd167fc0391fd4d11ec6bb4a3763","versionId":"","sequencer":"C7C7F864F0A0C306","metadata":[],"tags":[]}},"eventId":"1694025671.113484.56b2dd167fc0391fd4d11ec6bb4a3763","opaqueData":""}]}' 2023-09-06T18:54:38.636 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.636 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:10.943327Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.17355266024743407755","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"6","size":3008365,"eTag":"088972382ea42f90d987f2b34394390e","versionId":"","sequencer":"C7C7F86475B4C306","metadata":[],"tags":[]}},"eventId":"1694025671.113489.088972382ea42f90d987f2b34394390e","opaqueData":""}]}' 2023-09-06T18:54:38.636 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/23 2023-09-06T18:54:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/23 2023-09-06T18:54:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/23 2023-09-06T18:54:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'wIK/Edr5KypN/1KX6SmSxw==', 'Content-Length': '3010312', 'Expect': '100-Continue'} 2023-09-06T18:54:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.638 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.638 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.638 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.638 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.638 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.639 INFO:teuthology.orchestra.run.smithi099.stderr:wIK/Edr5KypN/1KX6SmSxw== 2023-09-06T18:54:38.639 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.639 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.639 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/23 2023-09-06T18:54:38.639 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.639 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:ZAWLy5WdQXi6JXxfTUJIwS/qnPA= 2023-09-06T18:54:38.640 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'wIK/Edr5KypN/1KX6SmSxw==', 'Content-Length': '3010312', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:ZAWLy5WdQXi6JXxfTUJIwS/qnPA='} 2023-09-06T18:54:38.640 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/24 2023-09-06T18:54:38.640 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/24 2023-09-06T18:54:38.640 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.640 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/24 2023-09-06T18:54:38.640 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.641 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'SyW/GgZZzzPOrXlM4vmG6g==', 'Content-Length': '3009940', 'Expect': '100-Continue'} 2023-09-06T18:54:38.641 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.641 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.641 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.641 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.641 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.642 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.642 INFO:teuthology.orchestra.run.smithi099.stderr:SyW/GgZZzzPOrXlM4vmG6g== 2023-09-06T18:54:38.642 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.642 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.642 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/24 2023-09-06T18:54:38.643 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.643 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:qiTQD7um5NcBlNElZewgkHrfVxk= 2023-09-06T18:54:38.643 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'SyW/GgZZzzPOrXlM4vmG6g==', 'Content-Length': '3009940', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:qiTQD7um5NcBlNElZewgkHrfVxk='} 2023-09-06T18:54:38.643 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/25 2023-09-06T18:54:38.643 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/25 2023-09-06T18:54:38.643 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.644 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/25 2023-09-06T18:54:38.644 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.644 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'VPBPP2XgMrGKGe0uJw8vyA==', 'Content-Length': '3007960', 'Expect': '100-Continue'} 2023-09-06T18:54:38.644 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.644 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.644 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:VPBPP2XgMrGKGe0uJw8vyA== 2023-09-06T18:54:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.646 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/25 2023-09-06T18:54:38.646 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.646 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:A3he1cu/npaVDcEg9dXQyZzs1BA= 2023-09-06T18:54:38.646 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'VPBPP2XgMrGKGe0uJw8vyA==', 'Content-Length': '3007960', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:A3he1cu/npaVDcEg9dXQyZzs1BA='} 2023-09-06T18:54:38.647 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/26 2023-09-06T18:54:38.647 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/26 2023-09-06T18:54:38.647 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.647 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/26 2023-09-06T18:54:38.647 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.647 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'JFQYXZM6xVePpuuTf602Qw==', 'Content-Length': '3010199', 'Expect': '100-Continue'} 2023-09-06T18:54:38.648 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.648 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.648 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.648 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.648 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.648 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.649 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.649 INFO:teuthology.orchestra.run.smithi099.stderr:JFQYXZM6xVePpuuTf602Qw== 2023-09-06T18:54:38.649 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.649 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.649 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/26 2023-09-06T18:54:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:zfrd15kkDIR+g3abNlyiZb9mioA= 2023-09-06T18:54:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'JFQYXZM6xVePpuuTf602Qw==', 'Content-Length': '3010199', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:zfrd15kkDIR+g3abNlyiZb9mioA='} 2023-09-06T18:54:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/27 2023-09-06T18:54:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/27 2023-09-06T18:54:38.651 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.651 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/27 2023-09-06T18:54:38.651 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.651 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'YsCPEqeLP86YZHFHlibJdg==', 'Content-Length': '3012319', 'Expect': '100-Continue'} 2023-09-06T18:54:38.651 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.653 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.653 INFO:teuthology.orchestra.run.smithi099.stderr:YsCPEqeLP86YZHFHlibJdg== 2023-09-06T18:54:38.653 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.653 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.653 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/27 2023-09-06T18:54:38.653 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.654 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Oz4ni+1XTsP/UKkTZETj3XBKlBI= 2023-09-06T18:54:38.654 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'YsCPEqeLP86YZHFHlibJdg==', 'Content-Length': '3012319', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Oz4ni+1XTsP/UKkTZETj3XBKlBI='} 2023-09-06T18:54:38.654 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"088972382ea42f90d987f2b34394390e"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f0da4a7f533c908b-0064f8c7c6-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.654 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"56b2dd167fc0391fd4d11ec6bb4a3763"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003afd0a3889609da2-0064f8c7c6-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.654 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"b4732af11051ff7c66788e4936bae690"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000957110928e2c26ad-0064f8c7c6-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.654 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.655 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.655 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.032044Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.6401212051087397035","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"8","size":3010227,"eTag":"db9ebe66fdffe24306e407e2faf789ab","versionId":"","sequencer":"C7C7F864EB0BBC0C","metadata":[],"tags":[]}},"eventId":"1694025671.213650.db9ebe66fdffe24306e407e2faf789ab","opaqueData":""}]}' 2023-09-06T18:54:38.655 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.655 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.065031Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.3982984264661256794","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"9","size":3008592,"eTag":"4e27e36340d1c2164a0f6fee2132b1cc","versionId":"","sequencer":"C7C7F864AA14BC0C","metadata":[],"tags":[]}},"eventId":"1694025671.213652.4e27e36340d1c2164a0f6fee2132b1cc","opaqueData":""}]}' 2023-09-06T18:54:38.655 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.656 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.097796Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.1071699808165623007","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"10","size":3010335,"eTag":"5c7430ac5c82fe30e4af11d01d4f2179","versionId":"","sequencer":"C7C7F8648D0DBC0C","metadata":[],"tags":[]}},"eventId":"1694025671.213650.5c7430ac5c82fe30e4af11d01d4f2179","opaqueData":""}]}' 2023-09-06T18:54:38.656 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.656 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.656 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.253273Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.16614366580783045038","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"15","size":3011089,"eTag":"d6f56b7e89b909dde353ddfb0d06fff9","versionId":"","sequencer":"C7C7F8649D2CA40F","metadata":[],"tags":[]}},"eventId":"1694025671.262417.d6f56b7e89b909dde353ddfb0d06fff9","opaqueData":""}]}' 2023-09-06T18:54:38.656 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.657 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.269047Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.14406552269597526279","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"14","size":3010514,"eTag":"cda4485c9c94e776e210d1688ab550d0","versionId":"","sequencer":"C7C7F86418C4C710","metadata":[],"tags":[]}},"eventId":"1694025671.281527.cda4485c9c94e776e210d1688ab550d0","opaqueData":""}]}' 2023-09-06T18:54:38.657 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/28 2023-09-06T18:54:38.657 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/28 2023-09-06T18:54:38.657 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.657 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/28 2023-09-06T18:54:38.657 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.658 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'q1ZgTvV58mn7MP47StSz2w==', 'Content-Length': '3008028', 'Expect': '100-Continue'} 2023-09-06T18:54:38.658 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.658 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.658 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.658 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.659 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.659 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.659 INFO:teuthology.orchestra.run.smithi099.stderr:q1ZgTvV58mn7MP47StSz2w== 2023-09-06T18:54:38.659 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.659 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.659 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/28 2023-09-06T18:54:38.660 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.660 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:5cUSpNdOFWbN+sfq5wmp9ncxhm0= 2023-09-06T18:54:38.660 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'q1ZgTvV58mn7MP47StSz2w==', 'Content-Length': '3008028', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:5cUSpNdOFWbN+sfq5wmp9ncxhm0='} 2023-09-06T18:54:38.660 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/29 2023-09-06T18:54:38.660 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/29 2023-09-06T18:54:38.661 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.661 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/29 2023-09-06T18:54:38.661 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.661 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'OX6QCjLG9OPI9p08qnRLgw==', 'Content-Length': '3011897', 'Expect': '100-Continue'} 2023-09-06T18:54:38.661 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.661 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.662 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.662 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.662 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.662 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.662 INFO:teuthology.orchestra.run.smithi099.stderr:OX6QCjLG9OPI9p08qnRLgw== 2023-09-06T18:54:38.663 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.663 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.663 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/29 2023-09-06T18:54:38.663 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.663 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:QNDapM1QeKpJS6V4k/MXivb31kk= 2023-09-06T18:54:38.664 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'OX6QCjLG9OPI9p08qnRLgw==', 'Content-Length': '3011897', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:QNDapM1QeKpJS6V4k/MXivb31kk='} 2023-09-06T18:54:38.664 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/31 2023-09-06T18:54:38.664 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/31 2023-09-06T18:54:38.664 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.664 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/31 2023-09-06T18:54:38.664 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.665 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '89LxHbh4U0prCKaZaxkGwA==', 'Content-Length': '3006647', 'Expect': '100-Continue'} 2023-09-06T18:54:38.665 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.665 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.665 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.665 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.666 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.666 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.666 INFO:teuthology.orchestra.run.smithi099.stderr:89LxHbh4U0prCKaZaxkGwA== 2023-09-06T18:54:38.666 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.666 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.666 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/31 2023-09-06T18:54:38.667 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.667 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:br3uwFfCdwFrnJp2CUkBfDtt/4M= 2023-09-06T18:54:38.667 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '89LxHbh4U0prCKaZaxkGwA==', 'Content-Length': '3006647', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:br3uwFfCdwFrnJp2CUkBfDtt/4M='} 2023-09-06T18:54:38.667 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/30 2023-09-06T18:54:38.667 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/30 2023-09-06T18:54:38.668 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.668 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/30 2023-09-06T18:54:38.668 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.668 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'uAfwq3FFt6p25lk5nOQtpA==', 'Content-Length': '3010060', 'Expect': '100-Continue'} 2023-09-06T18:54:38.668 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.669 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.669 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.669 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.669 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.669 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.669 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.670 INFO:teuthology.orchestra.run.smithi099.stderr:uAfwq3FFt6p25lk5nOQtpA== 2023-09-06T18:54:38.670 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.670 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.670 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/30 2023-09-06T18:54:38.671 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.671 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:z7F0/DuxBYIHCMD4ZjmXqhCFn0I= 2023-09-06T18:54:38.671 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'uAfwq3FFt6p25lk5nOQtpA==', 'Content-Length': '3010060', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:z7F0/DuxBYIHCMD4ZjmXqhCFn0I='} 2023-09-06T18:54:38.671 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"4e27e36340d1c2164a0f6fee2132b1cc"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000037466716011aca5a-0064f8c7c6-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.671 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"db9ebe66fdffe24306e407e2faf789ab"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000058d5ac72050390ab-0064f8c7c6-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.672 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.672 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.672 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.142108Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.2757882580617200765","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"11","size":3011165,"eTag":"027328bd50956a60d8e11483fe534139","versionId":"","sequencer":"C7C7F8645DD5B412","metadata":[],"tags":[]}},"eventId":"1694025671.313840.027328bd50956a60d8e11483fe534139","opaqueData":""}]}' 2023-09-06T18:54:38.672 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.673 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.159274Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11696590955431901269","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"12","size":3010730,"eTag":"dc1650c757437d3660bff8a08919a72d","versionId":"","sequencer":"C7C7F8645EDBB412","metadata":[],"tags":[]}},"eventId":"1694025671.313842.dc1650c757437d3660bff8a08919a72d","opaqueData":""}]}' 2023-09-06T18:54:38.673 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.673 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.189808Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9394318104517758579","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"13","size":3007999,"eTag":"ed694968e90e94699459568ee13138c7","versionId":"","sequencer":"C7C7F8641440B512","metadata":[],"tags":[]}},"eventId":"1694025671.313868.ed694968e90e94699459568ee13138c7","opaqueData":""}]}' 2023-09-06T18:54:38.673 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"5c7430ac5c82fe30e4af11d01d4f2179"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000edf714c2390f4df-0064f8c7c6-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.673 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.674 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.674 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.302296Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.16929041926730838654","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"16","size":3009990,"eTag":"4afaed905479f024163d26d697931ed8","versionId":"","sequencer":"C7C7F86477744D13","metadata":[],"tags":[]}},"eventId":"1694025671.323843.4afaed905479f024163d26d697931ed8","opaqueData":""}]}' 2023-09-06T18:54:38.674 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.674 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.316033Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11393747005530562645","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"17","size":3009758,"eTag":"33a3059712331ba02b054a9bbc1f2943","versionId":"","sequencer":"C7C7F8643103B913","metadata":[],"tags":[]}},"eventId":"1694025671.330892.33a3059712331ba02b054a9bbc1f2943","opaqueData":""}]}' 2023-09-06T18:54:38.674 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.675 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.380055Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11868492695156375452","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"18","size":3010849,"eTag":"2f2cbff395482704e83a791c6c6555ef","versionId":"","sequencer":"C7C7F864CEEF6F17","metadata":[],"tags":[]}},"eventId":"1694025671.393211.2f2cbff395482704e83a791c6c6555ef","opaqueData":""}]}' 2023-09-06T18:54:38.675 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.675 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.377508Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.12396651855725158552","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"19","size":3012251,"eTag":"6e0827baab321c4ea05a3f2620533e6c","versionId":"","sequencer":"C7C7F864BF9E9B17","metadata":[],"tags":[]}},"eventId":"1694025671.396074.6e0827baab321c4ea05a3f2620533e6c","opaqueData":""}]}' 2023-09-06T18:54:38.675 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/32 2023-09-06T18:54:38.675 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/32 2023-09-06T18:54:38.676 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.676 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/32 2023-09-06T18:54:38.676 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.676 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '6HvlNp8fFS4XQId3gafaOg==', 'Content-Length': '3013445', 'Expect': '100-Continue'} 2023-09-06T18:54:38.676 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.676 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.677 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.677 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.677 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.677 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.677 INFO:teuthology.orchestra.run.smithi099.stderr:6HvlNp8fFS4XQId3gafaOg== 2023-09-06T18:54:38.677 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.678 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.678 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/32 2023-09-06T18:54:38.678 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.678 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:WLYexgJdK6m9c76GQClp4hGbBNM= 2023-09-06T18:54:38.678 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '6HvlNp8fFS4XQId3gafaOg==', 'Content-Length': '3013445', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:WLYexgJdK6m9c76GQClp4hGbBNM='} 2023-09-06T18:54:38.679 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/33 2023-09-06T18:54:38.679 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/33 2023-09-06T18:54:38.679 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.679 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/33 2023-09-06T18:54:38.679 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.679 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Oaiyk69iqIEDaDuY6RzpOQ==', 'Content-Length': '3011199', 'Expect': '100-Continue'} 2023-09-06T18:54:38.680 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.680 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.680 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.680 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.680 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.681 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.681 INFO:teuthology.orchestra.run.smithi099.stderr:Oaiyk69iqIEDaDuY6RzpOQ== 2023-09-06T18:54:38.681 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.681 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.681 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/33 2023-09-06T18:54:38.681 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.682 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VwKG+mKnvTv7VYUzO1bXcwWTy68= 2023-09-06T18:54:38.682 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Oaiyk69iqIEDaDuY6RzpOQ==', 'Content-Length': '3011199', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VwKG+mKnvTv7VYUzO1bXcwWTy68='} 2023-09-06T18:54:38.682 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/34 2023-09-06T18:54:38.682 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/34 2023-09-06T18:54:38.683 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.683 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/34 2023-09-06T18:54:38.683 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.683 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Wqr6zPPTBYBl4QgTEBce6Q==', 'Content-Length': '3007945', 'Expect': '100-Continue'} 2023-09-06T18:54:38.683 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.683 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.684 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.684 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.684 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.684 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.684 INFO:teuthology.orchestra.run.smithi099.stderr:Wqr6zPPTBYBl4QgTEBce6Q== 2023-09-06T18:54:38.684 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.685 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.685 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/34 2023-09-06T18:54:38.685 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.685 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:4pk9N8uxce/2DnvMsLCmcty19Js= 2023-09-06T18:54:38.685 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Wqr6zPPTBYBl4QgTEBce6Q==', 'Content-Length': '3007945', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:4pk9N8uxce/2DnvMsLCmcty19Js='} 2023-09-06T18:54:38.686 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/35 2023-09-06T18:54:38.686 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/35 2023-09-06T18:54:38.686 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.686 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/35 2023-09-06T18:54:38.686 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.686 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Dqy8JCLcZ3AeSCSzwa1uUg==', 'Content-Length': '3012229', 'Expect': '100-Continue'} 2023-09-06T18:54:38.687 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.687 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.687 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.687 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.687 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.687 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.688 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.688 INFO:teuthology.orchestra.run.smithi099.stderr:Dqy8JCLcZ3AeSCSzwa1uUg== 2023-09-06T18:54:38.688 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.688 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.688 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/35 2023-09-06T18:54:38.689 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.689 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6ZiCj/7enk4oThQSWAcN8ZVAuNk= 2023-09-06T18:54:38.689 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Dqy8JCLcZ3AeSCSzwa1uUg==', 'Content-Length': '3012229', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6ZiCj/7enk4oThQSWAcN8ZVAuNk='} 2023-09-06T18:54:38.689 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/36 2023-09-06T18:54:38.689 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/36 2023-09-06T18:54:38.689 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.690 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/36 2023-09-06T18:54:38.690 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.690 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'jug51IfveFTwAxvvwiZZeQ==', 'Content-Length': '3010588', 'Expect': '100-Continue'} 2023-09-06T18:54:38.690 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.690 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.691 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.691 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.691 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.691 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.691 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.691 INFO:teuthology.orchestra.run.smithi099.stderr:jug51IfveFTwAxvvwiZZeQ== 2023-09-06T18:54:38.692 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.692 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.692 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/36 2023-09-06T18:54:38.692 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.692 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:MZz+FJGX35BeJJkuE2+5DC8w1bw= 2023-09-06T18:54:38.693 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'jug51IfveFTwAxvvwiZZeQ==', 'Content-Length': '3010588', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:MZz+FJGX35BeJJkuE2+5DC8w1bw='} 2023-09-06T18:54:38.693 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.693 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.693 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.269047Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.14406552269597526279","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"14","size":3010514,"eTag":"cda4485c9c94e776e210d1688ab550d0","versionId":"","sequencer":"C7C7F86418A6AD18","metadata":[],"tags":[]}},"eventId":"1694025671.414033.cda4485c9c94e776e210d1688ab550d0","opaqueData":""}]}' 2023-09-06T18:54:38.693 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.694 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.253273Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.16614366580783045038","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"15","size":3011089,"eTag":"d6f56b7e89b909dde353ddfb0d06fff9","versionId":"","sequencer":"C7C7F864C8A3AD18","metadata":[],"tags":[]}},"eventId":"1694025671.414032.d6f56b7e89b909dde353ddfb0d06fff9","opaqueData":""}]}' 2023-09-06T18:54:38.694 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"027328bd50956a60d8e11483fe534139"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000002645f7985ded047d-0064f8c7c6-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.694 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"ed694968e90e94699459568ee13138c7"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000825f528af6cd4e73-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.694 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.694 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.695 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.428485Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9647627688707131949","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"21","size":3009927,"eTag":"7635effc365f760017c2ab464c3e0ee3","versionId":"","sequencer":"C7C7F8642A3F5C1A","metadata":[],"tags":[]}},"eventId":"1694025671.442253.7635effc365f760017c2ab464c3e0ee3","opaqueData":""}]}' 2023-09-06T18:54:38.695 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.695 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.458287Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.16884338614545752409","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"20","size":3010394,"eTag":"2eb1b78f853798169024d2c97d2c6a23","versionId":"","sequencer":"C7C7F86498B77D1C","metadata":[],"tags":[]}},"eventId":"1694025671.478001.2eb1b78f853798169024d2c97d2c6a23","opaqueData":""}]}' 2023-09-06T18:54:38.695 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"dc1650c757437d3660bff8a08919a72d"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a252a380b590b855-0064f8c7c6-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.695 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/37 2023-09-06T18:54:38.696 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/37 2023-09-06T18:54:38.696 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.696 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/37 2023-09-06T18:54:38.696 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.696 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'bpRzQ8eRM4UvawGAtl5yWw==', 'Content-Length': '3006691', 'Expect': '100-Continue'} 2023-09-06T18:54:38.696 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.697 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.697 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.697 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.697 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.697 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:bpRzQ8eRM4UvawGAtl5yWw== 2023-09-06T18:54:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/37 2023-09-06T18:54:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.699 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:flmDPl46uY6mLFT9FOguGcwI2MQ= 2023-09-06T18:54:38.699 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'bpRzQ8eRM4UvawGAtl5yWw==', 'Content-Length': '3006691', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:flmDPl46uY6mLFT9FOguGcwI2MQ='} 2023-09-06T18:54:38.699 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/38 2023-09-06T18:54:38.699 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/38 2023-09-06T18:54:38.699 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.700 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/38 2023-09-06T18:54:38.700 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.700 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'LY/Ju4qfS6ST3wn/5iRVfg==', 'Content-Length': '3011290', 'Expect': '100-Continue'} 2023-09-06T18:54:38.700 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:LY/Ju4qfS6ST3wn/5iRVfg== 2023-09-06T18:54:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/38 2023-09-06T18:54:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:GM8/D8EYveCxEfIf70rEJqMAQC0= 2023-09-06T18:54:38.703 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'LY/Ju4qfS6ST3wn/5iRVfg==', 'Content-Length': '3011290', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:GM8/D8EYveCxEfIf70rEJqMAQC0='} 2023-09-06T18:54:38.703 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/39 2023-09-06T18:54:38.703 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/39 2023-09-06T18:54:38.703 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.703 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/39 2023-09-06T18:54:38.704 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.704 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3swg0g56meiE0VRlIz0BXw==', 'Content-Length': '3006792', 'Expect': '100-Continue'} 2023-09-06T18:54:38.704 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.704 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.704 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.705 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.705 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.705 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.705 INFO:teuthology.orchestra.run.smithi099.stderr:3swg0g56meiE0VRlIz0BXw== 2023-09-06T18:54:38.705 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.705 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.706 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/39 2023-09-06T18:54:38.706 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.706 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:INVjBqRohayntzu0Fb4y+y0NyEE= 2023-09-06T18:54:38.706 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3swg0g56meiE0VRlIz0BXw==', 'Content-Length': '3006792', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:INVjBqRohayntzu0Fb4y+y0NyEE='} 2023-09-06T18:54:38.706 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/41 2023-09-06T18:54:38.707 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/41 2023-09-06T18:54:38.707 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.707 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/41 2023-09-06T18:54:38.707 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.707 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Z7ViTM/IEw9AUndt8d0PCQ==', 'Content-Length': '3010979', 'Expect': '100-Continue'} 2023-09-06T18:54:38.707 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.708 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.708 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.708 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.708 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.708 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.708 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.709 INFO:teuthology.orchestra.run.smithi099.stderr:Z7ViTM/IEw9AUndt8d0PCQ== 2023-09-06T18:54:38.709 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.709 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.709 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/41 2023-09-06T18:54:38.709 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.710 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:gQuDyKqoOnUr28v9MUfKoOdDV5U= 2023-09-06T18:54:38.710 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Z7ViTM/IEw9AUndt8d0PCQ==', 'Content-Length': '3010979', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:gQuDyKqoOnUr28v9MUfKoOdDV5U='} 2023-09-06T18:54:38.710 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/40 2023-09-06T18:54:38.710 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/40 2023-09-06T18:54:38.711 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.711 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/40 2023-09-06T18:54:38.711 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.711 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'NpcXnBqc73w1DqDbohKUgA==', 'Content-Length': '3009124', 'Expect': '100-Continue'} 2023-09-06T18:54:38.711 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.711 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.712 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.712 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.712 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.712 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.712 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.713 INFO:teuthology.orchestra.run.smithi099.stderr:NpcXnBqc73w1DqDbohKUgA== 2023-09-06T18:54:38.713 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.713 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.713 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/40 2023-09-06T18:54:38.714 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.714 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:RoJtJeENl0YKtwvexwywRoFD8Ks= 2023-09-06T18:54:38.714 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'NpcXnBqc73w1DqDbohKUgA==', 'Content-Length': '3009124', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:RoJtJeENl0YKtwvexwywRoFD8Ks='} 2023-09-06T18:54:38.714 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.714 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.715 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.316033Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11393747005530562645","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"17","size":3009758,"eTag":"33a3059712331ba02b054a9bbc1f2943","versionId":"","sequencer":"C7C7F8648A37A61E","metadata":[],"tags":[]}},"eventId":"1694025671.514209.33a3059712331ba02b054a9bbc1f2943","opaqueData":""}]}' 2023-09-06T18:54:38.715 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.715 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.302296Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.16929041926730838654","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"16","size":3009990,"eTag":"4afaed905479f024163d26d697931ed8","versionId":"","sequencer":"C7C7F864A13BA61E","metadata":[],"tags":[]}},"eventId":"1694025671.514210.4afaed905479f024163d26d697931ed8","opaqueData":""}]}' 2023-09-06T18:54:38.715 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.716 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.377508Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.12396651855725158552","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"19","size":3012251,"eTag":"6e0827baab321c4ea05a3f2620533e6c","versionId":"","sequencer":"C7C7F86463B6A61E","metadata":[],"tags":[]}},"eventId":"1694025671.514242.6e0827baab321c4ea05a3f2620533e6c","opaqueData":""}]}' 2023-09-06T18:54:38.716 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.716 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.380055Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11868492695156375452","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"18","size":3010849,"eTag":"2f2cbff395482704e83a791c6c6555ef","versionId":"","sequencer":"C7C7F8641237A61E","metadata":[],"tags":[]}},"eventId":"1694025671.514209.2f2cbff395482704e83a791c6c6555ef","opaqueData":""}]}' 2023-09-06T18:54:38.716 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"cda4485c9c94e776e210d1688ab550d0"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c7ee5af5c3754907-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.716 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.717 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.717 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.510258Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.725649822050138728","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"22","size":3010756,"eTag":"a11d41caaf2c40780d3b5c45b1ad2490","versionId":"","sequencer":"C7C7F86447891D1F","metadata":[],"tags":[]}},"eventId":"1694025671.522029.a11d41caaf2c40780d3b5c45b1ad2490","opaqueData":""}]}' 2023-09-06T18:54:38.717 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.717 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.528438Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9639987818405084889","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"23","size":3010312,"eTag":"c082bf11daf92b2a4dff5297e92992c7","versionId":"","sequencer":"C7C7F86442914D20","metadata":[],"tags":[]}},"eventId":"1694025671.541954.c082bf11daf92b2a4dff5297e92992c7","opaqueData":""}]}' 2023-09-06T18:54:38.717 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.718 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.545744Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.4469030494664044352","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"24","size":3009940,"eTag":"4b25bf1a0659cf33cead794ce2f986ea","versionId":"","sequencer":"C7C7F86440EA5021","metadata":[],"tags":[]}},"eventId":"1694025671.558950.4b25bf1a0659cf33cead794ce2f986ea","opaqueData":""}]}' 2023-09-06T18:54:38.718 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.718 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.575269Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.15857732766077653583","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"25","size":3007960,"eTag":"54f04f3f65e032b18a19ed2e270f2fc8","versionId":"","sequencer":"C7C7F864D975FD22","metadata":[],"tags":[]}},"eventId":"1694025671.587036.54f04f3f65e032b18a19ed2e270f2fc8","opaqueData":""}]}' 2023-09-06T18:54:38.718 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"d6f56b7e89b909dde353ddfb0d06fff9"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e6921660e7b1d9ae-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.718 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/43 2023-09-06T18:54:38.719 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/43 2023-09-06T18:54:38.719 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.719 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/43 2023-09-06T18:54:38.719 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.719 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'BW5zKEC8GXhlXj9/5vIFyA==', 'Content-Length': '3011116', 'Expect': '100-Continue'} 2023-09-06T18:54:38.720 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.720 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.720 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.721 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.721 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.721 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.721 INFO:teuthology.orchestra.run.smithi099.stderr:BW5zKEC8GXhlXj9/5vIFyA== 2023-09-06T18:54:38.721 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.722 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.722 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/43 2023-09-06T18:54:38.722 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.722 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:ynrlmQtyJ4mVf8CHAgqGz77ROxw= 2023-09-06T18:54:38.722 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'BW5zKEC8GXhlXj9/5vIFyA==', 'Content-Length': '3011116', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:ynrlmQtyJ4mVf8CHAgqGz77ROxw='} 2023-09-06T18:54:38.723 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/42 2023-09-06T18:54:38.723 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/42 2023-09-06T18:54:38.723 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.723 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/42 2023-09-06T18:54:38.723 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.724 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1a1AJ0P4Rm0gYEXzRmfEJQ==', 'Content-Length': '3009320', 'Expect': '100-Continue'} 2023-09-06T18:54:38.724 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.724 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.724 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.724 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.724 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.725 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.725 INFO:teuthology.orchestra.run.smithi099.stderr:1a1AJ0P4Rm0gYEXzRmfEJQ== 2023-09-06T18:54:38.725 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.725 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.725 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/42 2023-09-06T18:54:38.726 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.726 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Ds+icAOVh2NqqbEU5yRhMbm3ZvQ= 2023-09-06T18:54:38.726 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1a1AJ0P4Rm0gYEXzRmfEJQ==', 'Content-Length': '3009320', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Ds+icAOVh2NqqbEU5yRhMbm3ZvQ='} 2023-09-06T18:54:38.726 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/44 2023-09-06T18:54:38.726 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/44 2023-09-06T18:54:38.727 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.727 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/44 2023-09-06T18:54:38.727 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.727 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '9qwFpefEGf3eMaebQ4lSuw==', 'Content-Length': '3012112', 'Expect': '100-Continue'} 2023-09-06T18:54:38.727 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.727 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.728 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.728 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.728 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.728 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.728 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.729 INFO:teuthology.orchestra.run.smithi099.stderr:9qwFpefEGf3eMaebQ4lSuw== 2023-09-06T18:54:38.729 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.729 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.729 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/44 2023-09-06T18:54:38.729 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.729 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:tLTFHkSGjSCdDBHNxi6Qy/Hn1VQ= 2023-09-06T18:54:38.730 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '9qwFpefEGf3eMaebQ4lSuw==', 'Content-Length': '3012112', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:tLTFHkSGjSCdDBHNxi6Qy/Hn1VQ='} 2023-09-06T18:54:38.730 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/45 2023-09-06T18:54:38.730 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/45 2023-09-06T18:54:38.730 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.730 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/45 2023-09-06T18:54:38.731 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.731 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '4ggq4h5s8nndDPpyqkxj4A==', 'Content-Length': '3008312', 'Expect': '100-Continue'} 2023-09-06T18:54:38.731 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.731 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.731 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.732 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.732 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.732 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.732 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.732 INFO:teuthology.orchestra.run.smithi099.stderr:4ggq4h5s8nndDPpyqkxj4A== 2023-09-06T18:54:38.733 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.733 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.733 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/45 2023-09-06T18:54:38.733 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.733 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:I8+TfoQDPZbv44p5dhXQvDZDYH4= 2023-09-06T18:54:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '4ggq4h5s8nndDPpyqkxj4A==', 'Content-Length': '3008312', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:I8+TfoQDPZbv44p5dhXQvDZDYH4='} 2023-09-06T18:54:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"33a3059712331ba02b054a9bbc1f2943"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009e1eb888ed51e055-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"4afaed905479f024163d26d697931ed8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000eaf009f03e7c727e-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"6e0827baab321c4ea05a3f2620533e6c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ac09c12eb8af6098-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.735 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.629386Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.17343464347955705024","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"27","size":3012319,"eTag":"62c08f12a78b3fce986471479626c976","versionId":"","sequencer":"C7C7F864EAC44626","metadata":[],"tags":[]}},"eventId":"1694025671.642172.62c08f12a78b3fce986471479626c976","opaqueData":""}]}' 2023-09-06T18:54:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.735 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.629364Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.8850296442589401689","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"26","size":3010199,"eTag":"2454185d933ac5578fa6eb937fad3643","versionId":"","sequencer":"C7C7F86455BF9826","metadata":[],"tags":[]}},"eventId":"1694025671.647544.2454185d933ac5578fa6eb937fad3643","opaqueData":""}]}' 2023-09-06T18:54:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.735 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.661559Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11461306381935885347","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"28","size":3008028,"eTag":"ab56604ef579f269fb30fe3b4ad4b3db","versionId":"","sequencer":"C7C7F864DA113628","metadata":[],"tags":[]}},"eventId":"1694025671.674632.ab56604ef579f269fb30fe3b4ad4b3db","opaqueData":""}]}' 2023-09-06T18:54:38.736 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"2f2cbff395482704e83a791c6c6555ef"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a4b55b396a1d3f9c-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.736 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.736 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.736 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.458287Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.16884338614545752409","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"20","size":3010394,"eTag":"2eb1b78f853798169024d2c97d2c6a23","versionId":"","sequencer":"C7C7F864B8B59E24","metadata":[],"tags":[]}},"eventId":"1694025671.614380.2eb1b78f853798169024d2c97d2c6a23","opaqueData":""}]}' 2023-09-06T18:54:38.736 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.736 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.428485Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9647627688707131949","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"21","size":3009927,"eTag":"7635effc365f760017c2ab464c3e0ee3","versionId":"","sequencer":"C7C7F8649D999E24","metadata":[],"tags":[]}},"eventId":"1694025671.614373.7635effc365f760017c2ab464c3e0ee3","opaqueData":""}]}' 2023-09-06T18:54:38.737 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/47 2023-09-06T18:54:38.737 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/47 2023-09-06T18:54:38.737 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.737 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/47 2023-09-06T18:54:38.737 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.738 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'KS9Zw1n4Uy/4yzxtaxN67g==', 'Content-Length': '3011563', 'Expect': '100-Continue'} 2023-09-06T18:54:38.738 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.738 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.738 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.738 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.738 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.739 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.739 INFO:teuthology.orchestra.run.smithi099.stderr:KS9Zw1n4Uy/4yzxtaxN67g== 2023-09-06T18:54:38.739 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.739 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.739 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/47 2023-09-06T18:54:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:ry7ag8lCdyhucQzmaEhIIyJQk9U= 2023-09-06T18:54:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'KS9Zw1n4Uy/4yzxtaxN67g==', 'Content-Length': '3011563', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:ry7ag8lCdyhucQzmaEhIIyJQk9U='} 2023-09-06T18:54:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/46 2023-09-06T18:54:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/46 2023-09-06T18:54:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/46 2023-09-06T18:54:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'H7kJWcZU/nhYeVMbRYARZQ==', 'Content-Length': '3012953', 'Expect': '100-Continue'} 2023-09-06T18:54:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:H7kJWcZU/nhYeVMbRYARZQ== 2023-09-06T18:54:38.743 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.743 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.743 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/46 2023-09-06T18:54:38.743 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.743 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:26RAOXLBrijrwAKNVhgrv33I+Zk= 2023-09-06T18:54:38.743 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'H7kJWcZU/nhYeVMbRYARZQ==', 'Content-Length': '3012953', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:26RAOXLBrijrwAKNVhgrv33I+Zk='} 2023-09-06T18:54:38.744 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/48 2023-09-06T18:54:38.744 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/48 2023-09-06T18:54:38.744 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.744 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/48 2023-09-06T18:54:38.744 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GFtiSrNn8LefbCorOfhtwA==', 'Content-Length': '3010416', 'Expect': '100-Continue'} 2023-09-06T18:54:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:GFtiSrNn8LefbCorOfhtwA== 2023-09-06T18:54:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/48 2023-09-06T18:54:38.747 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.747 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:TVGh6yDkOV3dy7vdD3o77pfnmrA= 2023-09-06T18:54:38.747 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GFtiSrNn8LefbCorOfhtwA==', 'Content-Length': '3010416', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:TVGh6yDkOV3dy7vdD3o77pfnmrA='} 2023-09-06T18:54:38.747 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/49 2023-09-06T18:54:38.747 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/49 2023-09-06T18:54:38.748 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.748 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/49 2023-09-06T18:54:38.748 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.748 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'UOToxJ3OiOQVsozxYfjFeQ==', 'Content-Length': '3009028', 'Expect': '100-Continue'} 2023-09-06T18:54:38.748 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:UOToxJ3OiOQVsozxYfjFeQ== 2023-09-06T18:54:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/49 2023-09-06T18:54:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:ozOj+CSXCc1EIKTlT1DMjluIZcA= 2023-09-06T18:54:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'UOToxJ3OiOQVsozxYfjFeQ==', 'Content-Length': '3009028', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:ozOj+CSXCc1EIKTlT1DMjluIZcA='} 2023-09-06T18:54:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/50 2023-09-06T18:54:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/50 2023-09-06T18:54:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/50 2023-09-06T18:54:38.752 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.752 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '2J813NFiOO5GelQoB25vsg==', 'Content-Length': '3010308', 'Expect': '100-Continue'} 2023-09-06T18:54:38.752 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.752 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.752 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.753 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.753 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.753 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.753 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.753 INFO:teuthology.orchestra.run.smithi099.stderr:2J813NFiOO5GelQoB25vsg== 2023-09-06T18:54:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/50 2023-09-06T18:54:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:/8mXSASz4N0/hcD9k1bvBOme0wI= 2023-09-06T18:54:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '2J813NFiOO5GelQoB25vsg==', 'Content-Length': '3010308', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:/8mXSASz4N0/hcD9k1bvBOme0wI='} 2023-09-06T18:54:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/51 2023-09-06T18:54:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/51 2023-09-06T18:54:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/51 2023-09-06T18:54:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'DMzZ040tsscVFtljhbh5qw==', 'Content-Length': '3009659', 'Expect': '100-Continue'} 2023-09-06T18:54:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:DMzZ040tsscVFtljhbh5qw== 2023-09-06T18:54:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.758 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/51 2023-09-06T18:54:38.758 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.758 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:TFY/zM+En0aXYdh+yfcmqeflAyA= 2023-09-06T18:54:38.758 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'DMzZ040tsscVFtljhbh5qw==', 'Content-Length': '3009659', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:TFY/zM+En0aXYdh+yfcmqeflAyA='} 2023-09-06T18:54:38.761 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/52 2023-09-06T18:54:38.761 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/52 2023-09-06T18:54:38.761 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.761 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/52 2023-09-06T18:54:38.761 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.762 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'PzWirSjXaibqzib7MbF6Uw==', 'Content-Length': '3011074', 'Expect': '100-Continue'} 2023-09-06T18:54:38.762 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.762 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.762 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.762 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.763 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.763 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.763 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.763 INFO:teuthology.orchestra.run.smithi099.stderr:PzWirSjXaibqzib7MbF6Uw== 2023-09-06T18:54:38.764 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.764 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.764 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/52 2023-09-06T18:54:38.764 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.764 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Z+PQYusSeJrMLlKX7xUrW2LJjEY= 2023-09-06T18:54:38.764 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'PzWirSjXaibqzib7MbF6Uw==', 'Content-Length': '3011074', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Z+PQYusSeJrMLlKX7xUrW2LJjEY='} 2023-09-06T18:54:38.765 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.765 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.765 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.545744Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.4469030494664044352","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"24","size":3009940,"eTag":"4b25bf1a0659cf33cead794ce2f986ea","versionId":"","sequencer":"C7C7F864E5B6972A","metadata":[],"tags":[]}},"eventId":"1694025671.714585.4b25bf1a0659cf33cead794ce2f986ea","opaqueData":""}]}' 2023-09-06T18:54:38.765 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.765 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.510258Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.725649822050138728","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"22","size":3010756,"eTag":"a11d41caaf2c40780d3b5c45b1ad2490","versionId":"","sequencer":"C7C7F864A09D972A","metadata":[],"tags":[]}},"eventId":"1694025671.714579.a11d41caaf2c40780d3b5c45b1ad2490","opaqueData":""}]}' 2023-09-06T18:54:38.766 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.766 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.528438Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9639987818405084889","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"23","size":3010312,"eTag":"c082bf11daf92b2a4dff5297e92992c7","versionId":"","sequencer":"C7C7F86485B8972A","metadata":[],"tags":[]}},"eventId":"1694025671.714586.c082bf11daf92b2a4dff5297e92992c7","opaqueData":""}]}' 2023-09-06T18:54:38.766 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.766 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.575269Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.15857732766077653583","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"25","size":3007960,"eTag":"54f04f3f65e032b18a19ed2e270f2fc8","versionId":"","sequencer":"C7C7F8647ABD972A","metadata":[],"tags":[]}},"eventId":"1694025671.714587.54f04f3f65e032b18a19ed2e270f2fc8","opaqueData":""}]}' 2023-09-06T18:54:38.766 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"7635effc365f760017c2ab464c3e0ee3"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000085e342447e9c8a2d-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.767 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"2eb1b78f853798169024d2c97d2c6a23"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ea5138834b692159-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.767 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.767 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.767 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.727310Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9952187160755448647","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"31","size":3006647,"eTag":"f3d2f11db878534a6b08a6996b1906c0","versionId":"","sequencer":"C7C7F8647A264D2C","metadata":[],"tags":[]}},"eventId":"1694025671.743253.f3d2f11db878534a6b08a6996b1906c0","opaqueData":""}]}' 2023-09-06T18:54:38.768 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.768 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.745946Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9590497205867349057","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"29","size":3011897,"eTag":"397e900a32c6f4e3c8f69d3caa744b83","versionId":"","sequencer":"C7C7F86427260B2D","metadata":[],"tags":[]}},"eventId":"1694025671.755705.397e900a32c6f4e3c8f69d3caa744b83","opaqueData":""}]}' 2023-09-06T18:54:38.768 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.768 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.759410Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.14387029173325574266","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"30","size":3010060,"eTag":"b807f0ab7145b7aa76e659399ce42da4","versionId":"","sequencer":"C7C7F864BF02012E","metadata":[],"tags":[]}},"eventId":"1694025671.771818.b807f0ab7145b7aa76e659399ce42da4","opaqueData":""}]}' 2023-09-06T18:54:38.769 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/54 2023-09-06T18:54:38.769 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/54 2023-09-06T18:54:38.769 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.769 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/54 2023-09-06T18:54:38.769 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'AHLKVAukcBmKVWjmDXhmYQ==', 'Content-Length': '3010997', 'Expect': '100-Continue'} 2023-09-06T18:54:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.771 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.771 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.771 INFO:teuthology.orchestra.run.smithi099.stderr:AHLKVAukcBmKVWjmDXhmYQ== 2023-09-06T18:54:38.771 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.771 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/54 2023-09-06T18:54:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:5ZQxZcj/AiWPB1dJLpRQg9hv/S8= 2023-09-06T18:54:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'AHLKVAukcBmKVWjmDXhmYQ==', 'Content-Length': '3010997', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:5ZQxZcj/AiWPB1dJLpRQg9hv/S8='} 2023-09-06T18:54:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/53 2023-09-06T18:54:38.773 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/53 2023-09-06T18:54:38.773 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.773 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/53 2023-09-06T18:54:38.773 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.773 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GeLikMrOw733TNGJ780Axw==', 'Content-Length': '3012434', 'Expect': '100-Continue'} 2023-09-06T18:54:38.773 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.774 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.774 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.774 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.774 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.774 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:GeLikMrOw733TNGJ780Axw== 2023-09-06T18:54:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/53 2023-09-06T18:54:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:eir29nzLxV+T2LYAocDhfD8nOSg= 2023-09-06T18:54:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GeLikMrOw733TNGJ780Axw==', 'Content-Length': '3012434', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:eir29nzLxV+T2LYAocDhfD8nOSg='} 2023-09-06T18:54:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/56 2023-09-06T18:54:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/56 2023-09-06T18:54:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/56 2023-09-06T18:54:38.777 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.777 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'BOKwCkRzWo0cfyfYPA2HwA==', 'Content-Length': '3012283', 'Expect': '100-Continue'} 2023-09-06T18:54:38.777 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.777 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.777 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.778 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.778 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.778 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.778 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.778 INFO:teuthology.orchestra.run.smithi099.stderr:BOKwCkRzWo0cfyfYPA2HwA== 2023-09-06T18:54:38.779 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.779 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.779 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/56 2023-09-06T18:54:38.779 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.779 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:IOIyM3tTOACM6AKEfeH0uk6RKqQ= 2023-09-06T18:54:38.779 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'BOKwCkRzWo0cfyfYPA2HwA==', 'Content-Length': '3012283', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:IOIyM3tTOACM6AKEfeH0uk6RKqQ='} 2023-09-06T18:54:38.780 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/55 2023-09-06T18:54:38.780 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/55 2023-09-06T18:54:38.780 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.780 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/55 2023-09-06T18:54:38.780 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.781 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HCUzvbk5ESoZuIKrZ+Wbhw==', 'Content-Length': '3007528', 'Expect': '100-Continue'} 2023-09-06T18:54:38.781 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.781 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.781 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.781 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.782 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.782 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.782 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.782 INFO:teuthology.orchestra.run.smithi099.stderr:HCUzvbk5ESoZuIKrZ+Wbhw== 2023-09-06T18:54:38.782 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.783 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.783 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/55 2023-09-06T18:54:38.783 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.783 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6FLW1Hg9ytQz3r40C1Cz+feMtjQ= 2023-09-06T18:54:38.783 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HCUzvbk5ESoZuIKrZ+Wbhw==', 'Content-Length': '3007528', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6FLW1Hg9ytQz3r40C1Cz+feMtjQ='} 2023-09-06T18:54:38.783 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/58 2023-09-06T18:54:38.784 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/58 2023-09-06T18:54:38.784 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.784 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/58 2023-09-06T18:54:38.784 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.784 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'FySTuawJnsPpemMPFgsmqw==', 'Content-Length': '3009470', 'Expect': '100-Continue'} 2023-09-06T18:54:38.785 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.785 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.785 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.785 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.785 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.786 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.786 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.786 INFO:teuthology.orchestra.run.smithi099.stderr:FySTuawJnsPpemMPFgsmqw== 2023-09-06T18:54:38.786 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.786 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.787 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/58 2023-09-06T18:54:38.787 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.787 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:4xeknzignwdIyJ+YRNTLAaxCwVA= 2023-09-06T18:54:38.787 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'FySTuawJnsPpemMPFgsmqw==', 'Content-Length': '3009470', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:4xeknzignwdIyJ+YRNTLAaxCwVA='} 2023-09-06T18:54:38.787 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"54f04f3f65e032b18a19ed2e270f2fc8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000dc11fbeed121924f-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.787 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"c082bf11daf92b2a4dff5297e92992c7"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000085c81dd89320bed9-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.788 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a11d41caaf2c40780d3b5c45b1ad2490"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000a1206ab229fba68-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.788 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.788 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.788 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.629364Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.8850296442589401689","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"26","size":3010199,"eTag":"2454185d933ac5578fa6eb937fad3643","versionId":"","sequencer":"C7C7F864EEDC9230","metadata":[],"tags":[]}},"eventId":"1694025671.814931.2454185d933ac5578fa6eb937fad3643","opaqueData":""}]}' 2023-09-06T18:54:38.789 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.789 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.661559Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11461306381935885347","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"28","size":3008028,"eTag":"ab56604ef579f269fb30fe3b4ad4b3db","versionId":"","sequencer":"C7C7F86481FB9230","metadata":[],"tags":[]}},"eventId":"1694025671.814939.ab56604ef579f269fb30fe3b4ad4b3db","opaqueData":""}]}' 2023-09-06T18:54:38.789 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.789 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.629386Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.17343464347955705024","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"27","size":3012319,"eTag":"62c08f12a78b3fce986471479626c976","versionId":"","sequencer":"C7C7F864C2DE9230","metadata":[],"tags":[]}},"eventId":"1694025671.814931.62c08f12a78b3fce986471479626c976","opaqueData":""}]}' 2023-09-06T18:54:38.789 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"4b25bf1a0659cf33cead794ce2f986ea"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003e052f8e94d6af40-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:11 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.790 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.790 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.790 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.818811Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.12072278786428593913","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"32","size":3013445,"eTag":"e87be5369f1f152e1740877781a7da3a","versionId":"","sequencer":"C7C7F86474ABA431","metadata":[],"tags":[]}},"eventId":"1694025671.832875.e87be5369f1f152e1740877781a7da3a","opaqueData":""}]}' 2023-09-06T18:54:38.790 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.790 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.834467Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.15160565427222440637","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"34","size":3007945,"eTag":"5aaafaccf3d3058065e1081310171ee9","versionId":"","sequencer":"C7C7F86444C75732","metadata":[],"tags":[]}},"eventId":"1694025671.844613.5aaafaccf3d3058065e1081310171ee9","opaqueData":""}]}' 2023-09-06T18:54:38.791 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.791 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.835751Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11235236320564160047","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"33","size":3011199,"eTag":"39a8b293af62a88103683b98e91ce939","versionId":"","sequencer":"C7C7F8647A7CBF32","metadata":[],"tags":[]}},"eventId":"1694025671.851410.39a8b293af62a88103683b98e91ce939","opaqueData":""}]}' 2023-09-06T18:54:38.791 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.791 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.873196Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.8875734717189488378","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"35","size":3012229,"eTag":"0eacbc2422dc67701e4824b3c1ad6e52","versionId":"","sequencer":"C7C7F86461ADB034","metadata":[],"tags":[]}},"eventId":"1694025671.883993.0eacbc2422dc67701e4824b3c1ad6e52","opaqueData":""}]}' 2023-09-06T18:54:38.791 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.791 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.871291Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.8727165782490815746","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"36","size":3010588,"eTag":"8ee839d487ef7854f0031befc2265979","versionId":"","sequencer":"C7C7F864FF28BF34","metadata":[],"tags":[]}},"eventId":"1694025671.884943.8ee839d487ef7854f0031befc2265979","opaqueData":""}]}' 2023-09-06T18:54:38.792 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.792 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.888214Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.16288818248913483059","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"37","size":3006691,"eTag":"6e947343c79133852f6b0180b65e725b","versionId":"","sequencer":"C7C7F864488AAC35","metadata":[],"tags":[]}},"eventId":"1694025671.900500.6e947343c79133852f6b0180b65e725b","opaqueData":""}]}' 2023-09-06T18:54:38.792 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/57 2023-09-06T18:54:38.792 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/57 2023-09-06T18:54:38.793 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.793 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/57 2023-09-06T18:54:38.793 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.793 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '94G9rxN9r7x8CGdidMFaAg==', 'Content-Length': '3010275', 'Expect': '100-Continue'} 2023-09-06T18:54:38.793 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.793 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:94G9rxN9r7x8CGdidMFaAg== 2023-09-06T18:54:38.795 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.795 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.795 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/57 2023-09-06T18:54:38.795 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.795 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6ayqm6Wsa7zRLf7hW2VRXbb1EfY= 2023-09-06T18:54:38.795 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '94G9rxN9r7x8CGdidMFaAg==', 'Content-Length': '3010275', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6ayqm6Wsa7zRLf7hW2VRXbb1EfY='} 2023-09-06T18:54:38.796 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/60 2023-09-06T18:54:38.796 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/60 2023-09-06T18:54:38.796 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.796 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/60 2023-09-06T18:54:38.796 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.797 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'FowqMMckz0VWUhuaNxe2nQ==', 'Content-Length': '3011279', 'Expect': '100-Continue'} 2023-09-06T18:54:38.797 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.797 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.797 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.797 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.798 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.798 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.798 INFO:teuthology.orchestra.run.smithi099.stderr:FowqMMckz0VWUhuaNxe2nQ== 2023-09-06T18:54:38.798 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.798 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.799 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/60 2023-09-06T18:54:38.799 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.799 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1HpIPoejfKGUqWjAYirHX5A9szI= 2023-09-06T18:54:38.799 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'FowqMMckz0VWUhuaNxe2nQ==', 'Content-Length': '3011279', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1HpIPoejfKGUqWjAYirHX5A9szI='} 2023-09-06T18:54:38.799 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/59 2023-09-06T18:54:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/59 2023-09-06T18:54:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/59 2023-09-06T18:54:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'dy8UZfYnVbeBuu0eEEEAdw==', 'Content-Length': '3013085', 'Expect': '100-Continue'} 2023-09-06T18:54:38.801 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.801 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.801 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.801 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.801 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.802 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.802 INFO:teuthology.orchestra.run.smithi099.stderr:dy8UZfYnVbeBuu0eEEEAdw== 2023-09-06T18:54:38.802 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.802 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.802 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/59 2023-09-06T18:54:38.803 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.803 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:FMb0hLbWXab8IMiyuTx2Sd1t5k4= 2023-09-06T18:54:38.803 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'dy8UZfYnVbeBuu0eEEEAdw==', 'Content-Length': '3013085', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:FMb0hLbWXab8IMiyuTx2Sd1t5k4='} 2023-09-06T18:54:38.803 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/61 2023-09-06T18:54:38.803 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/61 2023-09-06T18:54:38.804 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.804 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/61 2023-09-06T18:54:38.804 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.804 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'nLqB+IdmO3sqgi5tq0uz1g==', 'Content-Length': '3010350', 'Expect': '100-Continue'} 2023-09-06T18:54:38.804 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.804 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.805 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.805 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.805 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.805 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.805 INFO:teuthology.orchestra.run.smithi099.stderr:nLqB+IdmO3sqgi5tq0uz1g== 2023-09-06T18:54:38.806 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.806 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.806 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/61 2023-09-06T18:54:38.806 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.806 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:cmbvo+PadEKh0vQaqW+dgwHvbeQ= 2023-09-06T18:54:38.807 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'nLqB+IdmO3sqgi5tq0uz1g==', 'Content-Length': '3010350', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:cmbvo+PadEKh0vQaqW+dgwHvbeQ='} 2023-09-06T18:54:38.807 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/62 2023-09-06T18:54:38.807 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/62 2023-09-06T18:54:38.807 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.807 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/62 2023-09-06T18:54:38.807 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.808 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'L+CnbtRBIsmcKjWa1bgqAQ==', 'Content-Length': '3012829', 'Expect': '100-Continue'} 2023-09-06T18:54:38.808 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.808 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.808 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.808 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.809 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.809 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.809 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.809 INFO:teuthology.orchestra.run.smithi099.stderr:L+CnbtRBIsmcKjWa1bgqAQ== 2023-09-06T18:54:38.809 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.809 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:38.810 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/62 2023-09-06T18:54:38.810 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.810 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:YvmJ7d/T+PZLDfTssCi26WRDB6U= 2023-09-06T18:54:38.810 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'L+CnbtRBIsmcKjWa1bgqAQ==', 'Content-Length': '3012829', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:YvmJ7d/T+PZLDfTssCi26WRDB6U='} 2023-09-06T18:54:38.811 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"f3d2f11db878534a6b08a6996b1906c0"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008a1d457e67082b47-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.811 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"397e900a32c6f4e3c8f69d3caa744b83"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000085184a6567198441-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.811 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"2454185d933ac5578fa6eb937fad3643"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007ad291b94171d259-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.811 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"62c08f12a78b3fce986471479626c976"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f0b05cef528d34c0-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.811 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.812 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.812 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.989106Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.3983055690526323574","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"38","size":3011290,"eTag":"2d8fc9bb8a9f4ba493df09ffe624557e","versionId":"","sequencer":"C7C7F8644C47933B","metadata":[],"tags":[]}},"eventId":"1694025671.999507.2d8fc9bb8a9f4ba493df09ffe624557e","opaqueData":""}]}' 2023-09-06T18:54:38.812 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.812 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.999388Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.12691448335443453004","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"41","size":3010979,"eTag":"67b5624ccfc8130f4052776df1dd0f09","versionId":"","sequencer":"C8C7F8644862A500","metadata":[],"tags":[]}},"eventId":"1694025672.010838.67b5624ccfc8130f4052776df1dd0f09","opaqueData":""}]}' 2023-09-06T18:54:38.813 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"b807f0ab7145b7aa76e659399ce42da4"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c7a8feceb140b07a-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.813 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.813 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.813 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.745946Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9590497205867349057","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"29","size":3011897,"eTag":"397e900a32c6f4e3c8f69d3caa744b83","versionId":"","sequencer":"C7C7F86408D58C36","metadata":[],"tags":[]}},"eventId":"1694025671.915199.397e900a32c6f4e3c8f69d3caa744b83","opaqueData":""}]}' 2023-09-06T18:54:38.813 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.814 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.727310Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9952187160755448647","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"31","size":3006647,"eTag":"f3d2f11db878534a6b08a6996b1906c0","versionId":"","sequencer":"C7C7F8643BD58C36","metadata":[],"tags":[]}},"eventId":"1694025671.915199.f3d2f11db878534a6b08a6996b1906c0","opaqueData":""}]}' 2023-09-06T18:54:38.814 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.814 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.759410Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.14387029173325574266","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"30","size":3010060,"eTag":"b807f0ab7145b7aa76e659399ce42da4","versionId":"","sequencer":"C7C7F8643F3D8D36","metadata":[],"tags":[]}},"eventId":"1694025671.915225.b807f0ab7145b7aa76e659399ce42da4","opaqueData":""}]}' 2023-09-06T18:54:38.814 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"ab56604ef579f269fb30fe3b4ad4b3db"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009f0ebd6e0534bc23-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.814 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/63 2023-09-06T18:54:38.815 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/63 2023-09-06T18:54:38.815 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.815 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/63 2023-09-06T18:54:38.815 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.815 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'T5i766ZVJJ3eXXOUsc6lFA==', 'Content-Length': '3010173', 'Expect': '100-Continue'} 2023-09-06T18:54:38.816 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.816 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.816 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.816 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.816 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.816 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.817 INFO:teuthology.orchestra.run.smithi099.stderr:T5i766ZVJJ3eXXOUsc6lFA== 2023-09-06T18:54:38.817 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.817 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.817 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/63 2023-09-06T18:54:38.817 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.818 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:RYEZ13QtbXJqx9VanMv5lwSwvYo= 2023-09-06T18:54:38.818 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'T5i766ZVJJ3eXXOUsc6lFA==', 'Content-Length': '3010173', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:RYEZ13QtbXJqx9VanMv5lwSwvYo='} 2023-09-06T18:54:38.818 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/64 2023-09-06T18:54:38.818 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/64 2023-09-06T18:54:38.818 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/64 2023-09-06T18:54:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'sTGY538hYoLTXfpqTlI66Q==', 'Content-Length': '3012286', 'Expect': '100-Continue'} 2023-09-06T18:54:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.820 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.820 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.820 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.820 INFO:teuthology.orchestra.run.smithi099.stderr:sTGY538hYoLTXfpqTlI66Q== 2023-09-06T18:54:38.820 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/64 2023-09-06T18:54:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:kPQ7Nw0ABJAdPoKDwfRqYz6cZE0= 2023-09-06T18:54:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'sTGY538hYoLTXfpqTlI66Q==', 'Content-Length': '3012286', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:kPQ7Nw0ABJAdPoKDwfRqYz6cZE0='} 2023-09-06T18:54:38.822 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/65 2023-09-06T18:54:38.822 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/65 2023-09-06T18:54:38.822 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.822 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/65 2023-09-06T18:54:38.822 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.823 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'JY4+ecb2EmwJv//Rf3oChw==', 'Content-Length': '3010799', 'Expect': '100-Continue'} 2023-09-06T18:54:38.823 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.823 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.823 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.823 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.824 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.824 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.824 INFO:teuthology.orchestra.run.smithi099.stderr:JY4+ecb2EmwJv//Rf3oChw== 2023-09-06T18:54:38.824 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.825 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.825 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/65 2023-09-06T18:54:38.825 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.825 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:e7Brq42o6Lust4KBxlCx+CThsJ0= 2023-09-06T18:54:38.825 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'JY4+ecb2EmwJv//Rf3oChw==', 'Content-Length': '3010799', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:e7Brq42o6Lust4KBxlCx+CThsJ0='} 2023-09-06T18:54:38.826 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/67 2023-09-06T18:54:38.826 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/67 2023-09-06T18:54:38.826 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.826 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/67 2023-09-06T18:54:38.826 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.827 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'CExctBxSvJfXGKs/K/T9MQ==', 'Content-Length': '3011407', 'Expect': '100-Continue'} 2023-09-06T18:54:38.827 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.827 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.827 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.827 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.827 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:CExctBxSvJfXGKs/K/T9MQ== 2023-09-06T18:54:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/67 2023-09-06T18:54:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.829 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Kg+8r2tVv0ClHQIwVtGnpmy4f0A= 2023-09-06T18:54:38.829 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'CExctBxSvJfXGKs/K/T9MQ==', 'Content-Length': '3011407', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Kg+8r2tVv0ClHQIwVtGnpmy4f0A='} 2023-09-06T18:54:38.829 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/68 2023-09-06T18:54:38.829 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/68 2023-09-06T18:54:38.829 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.830 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/68 2023-09-06T18:54:38.830 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.830 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C6D+oxut6krwvUPn9E6aqQ==', 'Content-Length': '3009821', 'Expect': '100-Continue'} 2023-09-06T18:54:38.830 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.830 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.831 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.831 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.831 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.831 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.831 INFO:teuthology.orchestra.run.smithi099.stderr:C6D+oxut6krwvUPn9E6aqQ== 2023-09-06T18:54:38.832 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.832 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.832 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/68 2023-09-06T18:54:38.832 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.832 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:CtTT22ewDAnC9YahPSmJI1/2Xt0= 2023-09-06T18:54:38.833 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C6D+oxut6krwvUPn9E6aqQ==', 'Content-Length': '3009821', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:CtTT22ewDAnC9YahPSmJI1/2Xt0='} 2023-09-06T18:54:38.833 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.833 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.833 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.818811Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.12072278786428593913","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"32","size":3013445,"eTag":"e87be5369f1f152e1740877781a7da3a","versionId":"","sequencer":"C8C7F864D7EDF300","metadata":[],"tags":[]}},"eventId":"1694025672.015986.e87be5369f1f152e1740877781a7da3a","opaqueData":""}]}' 2023-09-06T18:54:38.833 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.834 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.873196Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.8875734717189488378","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"35","size":3012229,"eTag":"0eacbc2422dc67701e4824b3c1ad6e52","versionId":"","sequencer":"C8C7F8647701F400","metadata":[],"tags":[]}},"eventId":"1694025672.015991.0eacbc2422dc67701e4824b3c1ad6e52","opaqueData":""}]}' 2023-09-06T18:54:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.834 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.834467Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.15160565427222440637","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"34","size":3007945,"eTag":"5aaafaccf3d3058065e1081310171ee9","versionId":"","sequencer":"C8C7F864C3F7F300","metadata":[],"tags":[]}},"eventId":"1694025672.015988.5aaafaccf3d3058065e1081310171ee9","opaqueData":""}]}' 2023-09-06T18:54:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.834 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.835751Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11235236320564160047","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"33","size":3011199,"eTag":"39a8b293af62a88103683b98e91ce939","versionId":"","sequencer":"C8C7F8644C00F400","metadata":[],"tags":[]}},"eventId":"1694025672.015990.39a8b293af62a88103683b98e91ce939","opaqueData":""}]}' 2023-09-06T18:54:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.835 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.871291Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.8727165782490815746","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"36","size":3010588,"eTag":"8ee839d487ef7854f0031befc2265979","versionId":"","sequencer":"C8C7F864547EF400","metadata":[],"tags":[]}},"eventId":"1694025672.016023.8ee839d487ef7854f0031befc2265979","opaqueData":""}]}' 2023-09-06T18:54:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.835 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.888214Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.16288818248913483059","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"37","size":3006691,"eTag":"6e947343c79133852f6b0180b65e725b","versionId":"","sequencer":"C8C7F864FAC0F400","metadata":[],"tags":[]}},"eventId":"1694025672.016040.6e947343c79133852f6b0180b65e725b","opaqueData":""}]}' 2023-09-06T18:54:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.836 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.989106Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.3983055690526323574","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"38","size":3011290,"eTag":"2d8fc9bb8a9f4ba493df09ffe624557e","versionId":"","sequencer":"C8C7F864713CF500","metadata":[],"tags":[]}},"eventId":"1694025672.016071.2d8fc9bb8a9f4ba493df09ffe624557e","opaqueData":""}]}' 2023-09-06T18:54:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.836 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:11.999388Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.12691448335443453004","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"41","size":3010979,"eTag":"67b5624ccfc8130f4052776df1dd0f09","versionId":"","sequencer":"C8C7F8642E4DF500","metadata":[],"tags":[]}},"eventId":"1694025672.016076.67b5624ccfc8130f4052776df1dd0f09","opaqueData":""}]}' 2023-09-06T18:54:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.837 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.019925Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9382946917818381487","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"39","size":3006792,"eTag":"decc20d20e7a99e884d15465233d015f","versionId":"","sequencer":"C8C7F8646A8AE201","metadata":[],"tags":[]}},"eventId":"1694025672.031623.decc20d20e7a99e884d15465233d015f","opaqueData":""}]}' 2023-09-06T18:54:38.837 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.837 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.050468Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.3883934847856417374","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"40","size":3009124,"eTag":"3697179c1a9cef7c350ea0dba2129480","versionId":"","sequencer":"C8C7F8643DA0EE03","metadata":[],"tags":[]}},"eventId":"1694025672.065970.3697179c1a9cef7c350ea0dba2129480","opaqueData":""}]}' 2023-09-06T18:54:38.837 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.838 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.059709Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.3466641411800762714","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"43","size":3011116,"eTag":"056e732840bc1978655e3f7fe6f205c8","versionId":"","sequencer":"C8C7F8649E4D2B04","metadata":[],"tags":[]}},"eventId":"1694025672.069946.056e732840bc1978655e3f7fe6f205c8","opaqueData":""}]}' 2023-09-06T18:54:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/66 2023-09-06T18:54:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/66 2023-09-06T18:54:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/66 2023-09-06T18:54:38.839 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.839 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'H/7PVRaBhaiTgBGwsocA9g==', 'Content-Length': '3010803', 'Expect': '100-Continue'} 2023-09-06T18:54:38.839 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.839 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.839 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.839 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.840 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.840 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.840 INFO:teuthology.orchestra.run.smithi099.stderr:H/7PVRaBhaiTgBGwsocA9g== 2023-09-06T18:54:38.840 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.840 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.841 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/66 2023-09-06T18:54:38.841 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.841 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:36FUK1BTY8M1HxewVhurZeS63eU= 2023-09-06T18:54:38.841 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'H/7PVRaBhaiTgBGwsocA9g==', 'Content-Length': '3010803', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:36FUK1BTY8M1HxewVhurZeS63eU='} 2023-09-06T18:54:38.841 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/69 2023-09-06T18:54:38.842 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/69 2023-09-06T18:54:38.842 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.842 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/69 2023-09-06T18:54:38.842 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.842 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TloLB9lPBB2/TtlB2Gy1wA==', 'Content-Length': '3009018', 'Expect': '100-Continue'} 2023-09-06T18:54:38.843 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.843 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.843 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.843 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.843 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.844 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.844 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.844 INFO:teuthology.orchestra.run.smithi099.stderr:TloLB9lPBB2/TtlB2Gy1wA== 2023-09-06T18:54:38.844 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.844 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.844 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/69 2023-09-06T18:54:38.845 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.845 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6qcEOM3G1/zk11ieYYG76Z/6oiA= 2023-09-06T18:54:38.845 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TloLB9lPBB2/TtlB2Gy1wA==', 'Content-Length': '3009018', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6qcEOM3G1/zk11ieYYG76Z/6oiA='} 2023-09-06T18:54:38.845 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/70 2023-09-06T18:54:38.845 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/70 2023-09-06T18:54:38.846 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.846 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/70 2023-09-06T18:54:38.846 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.846 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ACjFwH5NGlKEdPWFDs0Uug==', 'Content-Length': '3011260', 'Expect': '100-Continue'} 2023-09-06T18:54:38.846 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.847 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.847 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.847 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.847 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.847 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.847 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:ACjFwH5NGlKEdPWFDs0Uug== 2023-09-06T18:54:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/70 2023-09-06T18:54:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.849 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Gvl3cUJ2iRK2IcsrZZrixV76LeQ= 2023-09-06T18:54:38.849 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ACjFwH5NGlKEdPWFDs0Uug==', 'Content-Length': '3011260', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Gvl3cUJ2iRK2IcsrZZrixV76LeQ='} 2023-09-06T18:54:38.849 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/71 2023-09-06T18:54:38.849 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/71 2023-09-06T18:54:38.849 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/71 2023-09-06T18:54:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'onB9dpXuID5Leh9ic6perg==', 'Content-Length': '3010257', 'Expect': '100-Continue'} 2023-09-06T18:54:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.852 INFO:teuthology.orchestra.run.smithi099.stderr:onB9dpXuID5Leh9ic6perg== 2023-09-06T18:54:38.852 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.852 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.852 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/71 2023-09-06T18:54:38.852 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.852 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Nu4fqXnsX73xXTrJxiVFnc1DqK8= 2023-09-06T18:54:38.853 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'onB9dpXuID5Leh9ic6perg==', 'Content-Length': '3010257', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Nu4fqXnsX73xXTrJxiVFnc1DqK8='} 2023-09-06T18:54:38.853 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"e87be5369f1f152e1740877781a7da3a"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a7895998368f66f9-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.853 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"39a8b293af62a88103683b98e91ce939"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009beb93e81ecb262f-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.853 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"0eacbc2422dc67701e4824b3c1ad6e52"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007b2cf1b31aae7efa-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"8ee839d487ef7854f0031befc2265979"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000791d1f0a694f1d02-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"67b5624ccfc8130f4052776df1dd0f09"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b0211504d754644c-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"5aaafaccf3d3058065e1081310171ee9"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d26525ee93a96abd-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"6e947343c79133852f6b0180b65e725b"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e20d81e541637533-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"2d8fc9bb8a9f4ba493df09ffe624557e"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003746a80c226b7376-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.855 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.118127Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9820474146093454851","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"42","size":3009320,"eTag":"d5ad402743f8466d206045f34667c425","versionId":"","sequencer":"C8C7F864BA019907","metadata":[],"tags":[]}},"eventId":"1694025672.127467.d5ad402743f8466d206045f34667c425","opaqueData":""}]}' 2023-09-06T18:54:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.855 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.127284Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.7438677205899361808","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"47","size":3011563,"eTag":"292f59c359f8532ff8cb3c6d6b137aee","versionId":"","sequencer":"C8C7F86482D29108","metadata":[],"tags":[]}},"eventId":"1694025672.143774.292f59c359f8532ff8cb3c6d6b137aee","opaqueData":""}]}' 2023-09-06T18:54:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.856 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.143551Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.18432459695270741403","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"44","size":3012112,"eTag":"f6ac05a5e7c419fdde31a79b438952bb","versionId":"","sequencer":"C8C7F864E50FD009","metadata":[],"tags":[]}},"eventId":"1694025672.164630.f6ac05a5e7c419fdde31a79b438952bb","opaqueData":""}]}' 2023-09-06T18:54:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.856 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.146480Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.15063916870826696958","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"46","size":3012953,"eTag":"1fb90959c654fe785879531b45801165","versionId":"","sequencer":"C8C7F8644B6FD209","metadata":[],"tags":[]}},"eventId":"1694025672.164785.1fb90959c654fe785879531b45801165","opaqueData":""}]}' 2023-09-06T18:54:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.857 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.138850Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.5489055613611098390","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"45","size":3008312,"eTag":"e2082ae21e6cf279dd0cfa72aa4c63e0","versionId":"","sequencer":"C8C7F864D371D209","metadata":[],"tags":[]}},"eventId":"1694025672.164786.e2082ae21e6cf279dd0cfa72aa4c63e0","opaqueData":""}]}' 2023-09-06T18:54:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.857 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.164816Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.185927170411413292","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"50","size":3010308,"eTag":"d89f35dcd16238ee467a5428076e6fb2","versionId":"","sequencer":"C8C7F864E9A8A40A","metadata":[],"tags":[]}},"eventId":"1694025672.178563.d89f35dcd16238ee467a5428076e6fb2","opaqueData":""}]}' 2023-09-06T18:54:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/72 2023-09-06T18:54:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/72 2023-09-06T18:54:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/72 2023-09-06T18:54:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '58/syNvnMEysIcHSAd8NBg==', 'Content-Length': '3012528', 'Expect': '100-Continue'} 2023-09-06T18:54:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.860 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.860 INFO:teuthology.orchestra.run.smithi099.stderr:58/syNvnMEysIcHSAd8NBg== 2023-09-06T18:54:38.860 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.860 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.860 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/72 2023-09-06T18:54:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:34itloabOLBXXfokpjOSbKGYcPY= 2023-09-06T18:54:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '58/syNvnMEysIcHSAd8NBg==', 'Content-Length': '3012528', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:34itloabOLBXXfokpjOSbKGYcPY='} 2023-09-06T18:54:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/74 2023-09-06T18:54:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/74 2023-09-06T18:54:38.862 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.862 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/74 2023-09-06T18:54:38.862 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.862 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'wH4O4G0cUa3RVZTZL8Bx8g==', 'Content-Length': '3009021', 'Expect': '100-Continue'} 2023-09-06T18:54:38.862 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.863 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.863 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.863 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.863 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.863 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.864 INFO:teuthology.orchestra.run.smithi099.stderr:wH4O4G0cUa3RVZTZL8Bx8g== 2023-09-06T18:54:38.864 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.864 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.864 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/74 2023-09-06T18:54:38.864 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.865 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:oXpzU2EWQ1vcOuvfW97TE3ZNzJg= 2023-09-06T18:54:38.865 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'wH4O4G0cUa3RVZTZL8Bx8g==', 'Content-Length': '3009021', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:oXpzU2EWQ1vcOuvfW97TE3ZNzJg='} 2023-09-06T18:54:38.865 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/73 2023-09-06T18:54:38.865 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/73 2023-09-06T18:54:38.865 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.866 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/73 2023-09-06T18:54:38.866 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.866 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ivwZ7Vdh/5AezzspgtVxkA==', 'Content-Length': '3008097', 'Expect': '100-Continue'} 2023-09-06T18:54:38.866 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.866 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.867 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.867 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.867 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.867 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.867 INFO:teuthology.orchestra.run.smithi099.stderr:ivwZ7Vdh/5AezzspgtVxkA== 2023-09-06T18:54:38.868 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.868 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.868 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/73 2023-09-06T18:54:38.868 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.868 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:rAgrUjCY/2UU1peWmmZ9tsZx824= 2023-09-06T18:54:38.869 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ivwZ7Vdh/5AezzspgtVxkA==', 'Content-Length': '3008097', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:rAgrUjCY/2UU1peWmmZ9tsZx824='} 2023-09-06T18:54:38.869 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/75 2023-09-06T18:54:38.869 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/75 2023-09-06T18:54:38.869 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.869 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/75 2023-09-06T18:54:38.870 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.870 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '13KPoJjsWZmb48Fg0K6R/g==', 'Content-Length': '3010936', 'Expect': '100-Continue'} 2023-09-06T18:54:38.870 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.870 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.870 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.871 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.871 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.871 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.871 INFO:teuthology.orchestra.run.smithi099.stderr:13KPoJjsWZmb48Fg0K6R/g== 2023-09-06T18:54:38.871 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.871 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.872 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/75 2023-09-06T18:54:38.872 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.872 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:8wq87cdlwIhv6AMzvkexpnLie1o= 2023-09-06T18:54:38.872 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '13KPoJjsWZmb48Fg0K6R/g==', 'Content-Length': '3010936', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:8wq87cdlwIhv6AMzvkexpnLie1o='} 2023-09-06T18:54:38.872 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/76 2023-09-06T18:54:38.873 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/76 2023-09-06T18:54:38.873 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.873 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/76 2023-09-06T18:54:38.873 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.873 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'O5q5DUvpDjeTWx2UQS0b/w==', 'Content-Length': '3012500', 'Expect': '100-Continue'} 2023-09-06T18:54:38.874 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.874 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.874 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.874 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.874 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.875 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.875 INFO:teuthology.orchestra.run.smithi099.stderr:O5q5DUvpDjeTWx2UQS0b/w== 2023-09-06T18:54:38.875 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.875 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.875 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/76 2023-09-06T18:54:38.876 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.876 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:klKXT0O66VTwU/oiBA1v0jgqpVU= 2023-09-06T18:54:38.876 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'O5q5DUvpDjeTWx2UQS0b/w==', 'Content-Length': '3012500', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:klKXT0O66VTwU/oiBA1v0jgqpVU='} 2023-09-06T18:54:38.876 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.876 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.877 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.211710Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.6630435446289497429","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"49","size":3009028,"eTag":"50e4e8c49dce88e415b28cf161f8c579","versionId":"","sequencer":"C8C7F864A440500D","metadata":[],"tags":[]}},"eventId":"1694025672.223363.50e4e8c49dce88e415b28cf161f8c579","opaqueData":""}]}' 2023-09-06T18:54:38.877 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.877 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.232206Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.2034980962141960969","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"48","size":3010416,"eTag":"185b624ab367f0b79f6c2a2b39f86dc0","versionId":"","sequencer":"C8C7F864A184840E","metadata":[],"tags":[]}},"eventId":"1694025672.243565.185b624ab367f0b79f6c2a2b39f86dc0","opaqueData":""}]}' 2023-09-06T18:54:38.877 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.877 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.271245Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11932110549230009864","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"51","size":3009659,"eTag":"0cccd9d38d2db2c71516d96385b879ab","versionId":"","sequencer":"C8C7F86478320111","metadata":[],"tags":[]}},"eventId":"1694025672.285291.0cccd9d38d2db2c71516d96385b879ab","opaqueData":""}]}' 2023-09-06T18:54:38.878 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.878 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.275918Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.7174042032887746701","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"52","size":3011074,"eTag":"3f35a2ad28d76a26eace26fb31b17a53","versionId":"","sequencer":"C8C7F86452C72F11","metadata":[],"tags":[]}},"eventId":"1694025672.288343.3f35a2ad28d76a26eace26fb31b17a53","opaqueData":""}]}' 2023-09-06T18:54:38.878 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.878 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.878 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.050468Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.3883934847856417374","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"40","size":3009124,"eTag":"3697179c1a9cef7c350ea0dba2129480","versionId":"","sequencer":"C8C7F8641934E70C","metadata":[],"tags":[]}},"eventId":"1694025672.216478.3697179c1a9cef7c350ea0dba2129480","opaqueData":""}]}' 2023-09-06T18:54:38.879 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.879 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.059709Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.3466641411800762714","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"43","size":3011116,"eTag":"056e732840bc1978655e3f7fe6f205c8","versionId":"","sequencer":"C8C7F864F135E70C","metadata":[],"tags":[]}},"eventId":"1694025672.216479.056e732840bc1978655e3f7fe6f205c8","opaqueData":""}]}' 2023-09-06T18:54:38.879 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.879 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.019925Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9382946917818381487","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"39","size":3006792,"eTag":"decc20d20e7a99e884d15465233d015f","versionId":"","sequencer":"C8C7F864A333E70C","metadata":[],"tags":[]}},"eventId":"1694025672.216478.decc20d20e7a99e884d15465233d015f","opaqueData":""}]}' 2023-09-06T18:54:38.879 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/78 2023-09-06T18:54:38.880 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/78 2023-09-06T18:54:38.880 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.880 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/78 2023-09-06T18:54:38.880 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.881 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'LInDI/4s7vtCmmUdl2GZgQ==', 'Content-Length': '3012430', 'Expect': '100-Continue'} 2023-09-06T18:54:38.881 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.881 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.881 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.881 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.882 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.882 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.882 INFO:teuthology.orchestra.run.smithi099.stderr:LInDI/4s7vtCmmUdl2GZgQ== 2023-09-06T18:54:38.882 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.882 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.882 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/78 2023-09-06T18:54:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:wTYx6jGw8NbNYPV0mbzgPssE+PE= 2023-09-06T18:54:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'LInDI/4s7vtCmmUdl2GZgQ==', 'Content-Length': '3012430', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:wTYx6jGw8NbNYPV0mbzgPssE+PE='} 2023-09-06T18:54:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/77 2023-09-06T18:54:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/77 2023-09-06T18:54:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/77 2023-09-06T18:54:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '8B3cc/t8TpRZF5cCer6dEQ==', 'Content-Length': '3008801', 'Expect': '100-Continue'} 2023-09-06T18:54:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.885 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.885 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.885 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.885 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.885 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.886 INFO:teuthology.orchestra.run.smithi099.stderr:8B3cc/t8TpRZF5cCer6dEQ== 2023-09-06T18:54:38.886 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.886 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.886 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/77 2023-09-06T18:54:38.886 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.887 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VWVCrMVG65GwMoPOxodu30Gx5IY= 2023-09-06T18:54:38.887 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '8B3cc/t8TpRZF5cCer6dEQ==', 'Content-Length': '3008801', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VWVCrMVG65GwMoPOxodu30Gx5IY='} 2023-09-06T18:54:38.887 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/79 2023-09-06T18:54:38.887 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/79 2023-09-06T18:54:38.887 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.888 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/79 2023-09-06T18:54:38.888 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.888 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'az8m3gLwA3rfuOuGoIUlIg==', 'Content-Length': '3009702', 'Expect': '100-Continue'} 2023-09-06T18:54:38.888 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.888 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:az8m3gLwA3rfuOuGoIUlIg== 2023-09-06T18:54:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.890 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.890 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/79 2023-09-06T18:54:38.890 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.890 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:g3Ny3rsaOzsW2WwvyD68FiQb99s= 2023-09-06T18:54:38.891 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'az8m3gLwA3rfuOuGoIUlIg==', 'Content-Length': '3009702', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:g3Ny3rsaOzsW2WwvyD68FiQb99s='} 2023-09-06T18:54:38.891 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/80 2023-09-06T18:54:38.891 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/80 2023-09-06T18:54:38.891 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.891 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/80 2023-09-06T18:54:38.892 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.892 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MILNOxKsGQkBzSCgDskNTA==', 'Content-Length': '3009370', 'Expect': '100-Continue'} 2023-09-06T18:54:38.892 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.892 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.892 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.893 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.893 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.893 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.893 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.893 INFO:teuthology.orchestra.run.smithi099.stderr:MILNOxKsGQkBzSCgDskNTA== 2023-09-06T18:54:38.894 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.894 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.894 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/80 2023-09-06T18:54:38.894 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.894 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:5sx0j0PgApPbvZFYmcsddlB3Dig= 2023-09-06T18:54:38.895 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MILNOxKsGQkBzSCgDskNTA==', 'Content-Length': '3009370', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:5sx0j0PgApPbvZFYmcsddlB3Dig='} 2023-09-06T18:54:38.895 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/82 2023-09-06T18:54:38.895 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/82 2023-09-06T18:54:38.895 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.895 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/82 2023-09-06T18:54:38.896 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.896 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'SCQrX/nmwhId+jfW+hbFKA==', 'Content-Length': '3013892', 'Expect': '100-Continue'} 2023-09-06T18:54:38.896 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.896 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.897 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.897 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.897 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.897 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.897 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.897 INFO:teuthology.orchestra.run.smithi099.stderr:SCQrX/nmwhId+jfW+hbFKA== 2023-09-06T18:54:38.898 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.898 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.898 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/82 2023-09-06T18:54:38.898 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.898 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:0g54EVFLjN5wdXtHGIks2sEB4/o= 2023-09-06T18:54:38.899 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'SCQrX/nmwhId+jfW+hbFKA==', 'Content-Length': '3013892', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:0g54EVFLjN5wdXtHGIks2sEB4/o='} 2023-09-06T18:54:38.899 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/81 2023-09-06T18:54:38.899 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/81 2023-09-06T18:54:38.899 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.899 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/81 2023-09-06T18:54:38.900 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.900 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TpxZegmN2A+sJ3sHtNR34g==', 'Content-Length': '3011797', 'Expect': '100-Continue'} 2023-09-06T18:54:38.900 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.900 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.901 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.901 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.901 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.901 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.901 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.901 INFO:teuthology.orchestra.run.smithi099.stderr:TpxZegmN2A+sJ3sHtNR34g== 2023-09-06T18:54:38.902 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.902 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.902 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/81 2023-09-06T18:54:38.902 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.902 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6EPFOpoyygYmIMj+Rug1f0pP8YM= 2023-09-06T18:54:38.903 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TpxZegmN2A+sJ3sHtNR34g==', 'Content-Length': '3011797', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6EPFOpoyygYmIMj+Rug1f0pP8YM='} 2023-09-06T18:54:38.903 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/84 2023-09-06T18:54:38.903 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/84 2023-09-06T18:54:38.903 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.903 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/84 2023-09-06T18:54:38.904 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.904 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Pynp8NrbQabMkK3jYh9YCQ==', 'Content-Length': '3010540', 'Expect': '100-Continue'} 2023-09-06T18:54:38.904 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.904 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.904 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.905 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.905 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.905 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.905 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.906 INFO:teuthology.orchestra.run.smithi099.stderr:Pynp8NrbQabMkK3jYh9YCQ== 2023-09-06T18:54:38.906 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.906 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.906 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/84 2023-09-06T18:54:38.906 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.907 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:DWb43RjPnXnEfLGJ6QXz3oIUtCk= 2023-09-06T18:54:38.907 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Pynp8NrbQabMkK3jYh9YCQ==', 'Content-Length': '3010540', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:DWb43RjPnXnEfLGJ6QXz3oIUtCk='} 2023-09-06T18:54:38.907 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/83 2023-09-06T18:54:38.907 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/83 2023-09-06T18:54:38.907 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.907 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/83 2023-09-06T18:54:38.908 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.908 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C/iQp/RnJcz00qfhYKVejg==', 'Content-Length': '3008624', 'Expect': '100-Continue'} 2023-09-06T18:54:38.908 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.908 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.908 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.909 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.909 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.909 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.909 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.909 INFO:teuthology.orchestra.run.smithi099.stderr:C/iQp/RnJcz00qfhYKVejg== 2023-09-06T18:54:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/83 2023-09-06T18:54:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VBQiPvpgkgY0eGav2GjAQBMnjks= 2023-09-06T18:54:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C/iQp/RnJcz00qfhYKVejg==', 'Content-Length': '3008624', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VBQiPvpgkgY0eGav2GjAQBMnjks='} 2023-09-06T18:54:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"decc20d20e7a99e884d15465233d015f"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008236ec823ee934af-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"3697179c1a9cef7c350ea0dba2129480"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000035e6822a893e1a5e-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"056e732840bc1978655e3f7fe6f205c8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000301bfbff3a0b095a-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.912 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.146480Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.15063916870826696958","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"46","size":3012953,"eTag":"1fb90959c654fe785879531b45801165","versionId":"","sequencer":"C8C7F8645523DF12","metadata":[],"tags":[]}},"eventId":"1694025672.316613.1fb90959c654fe785879531b45801165","opaqueData":""}]}' 2023-09-06T18:54:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.912 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.127284Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.7438677205899361808","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"47","size":3011563,"eTag":"292f59c359f8532ff8cb3c6d6b137aee","versionId":"","sequencer":"C8C7F864E5E1DE12","metadata":[],"tags":[]}},"eventId":"1694025672.316596.292f59c359f8532ff8cb3c6d6b137aee","opaqueData":""}]}' 2023-09-06T18:54:38.913 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.913 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.164816Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.185927170411413292","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"50","size":3010308,"eTag":"d89f35dcd16238ee467a5428076e6fb2","versionId":"","sequencer":"C8C7F86462A7DF12","metadata":[],"tags":[]}},"eventId":"1694025672.316647.d89f35dcd16238ee467a5428076e6fb2","opaqueData":""}]}' 2023-09-06T18:54:38.913 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.913 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.138850Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.5489055613611098390","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"45","size":3008312,"eTag":"e2082ae21e6cf279dd0cfa72aa4c63e0","versionId":"","sequencer":"C8C7F864016DDF12","metadata":[],"tags":[]}},"eventId":"1694025672.316632.e2082ae21e6cf279dd0cfa72aa4c63e0","opaqueData":""}]}' 2023-09-06T18:54:38.913 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.914 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.118127Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9820474146093454851","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"42","size":3009320,"eTag":"d5ad402743f8466d206045f34667c425","versionId":"","sequencer":"C8C7F8647AE2DE12","metadata":[],"tags":[]}},"eventId":"1694025672.316596.d5ad402743f8466d206045f34667c425","opaqueData":""}]}' 2023-09-06T18:54:38.914 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.914 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.143551Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.18432459695270741403","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"44","size":3012112,"eTag":"f6ac05a5e7c419fdde31a79b438952bb","versionId":"","sequencer":"C8C7F864C1E2DE12","metadata":[],"tags":[]}},"eventId":"1694025672.316596.f6ac05a5e7c419fdde31a79b438952bb","opaqueData":""}]}' 2023-09-06T18:54:38.914 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.914 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.915 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.308230Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.15038305735325146487","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"53","size":3012434,"eTag":"19e2e290cacec3bdf74cd189efcd00c7","versionId":"","sequencer":"C8C7F8641783ED12","metadata":[],"tags":[]}},"eventId":"1694025672.317555.19e2e290cacec3bdf74cd189efcd00c7","opaqueData":""}]}' 2023-09-06T18:54:38.915 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.915 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.336382Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.8596261825739537400","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"55","size":3007528,"eTag":"1c2533bdb939112a19b882ab67e59b87","versionId":"","sequencer":"C8C7F864FCE4BE14","metadata":[],"tags":[]}},"eventId":"1694025672.348054.1c2533bdb939112a19b882ab67e59b87","opaqueData":""}]}' 2023-09-06T18:54:38.915 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.915 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.368387Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.1696421320166965460","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"61","size":3010350,"eTag":"9cba81f887663b7b2a822e6dab4bb3d6","versionId":"","sequencer":"C8C7F86494EDAE16","metadata":[],"tags":[]}},"eventId":"1694025672.380562.9cba81f887663b7b2a822e6dab4bb3d6","opaqueData":""}]}' 2023-09-06T18:54:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.916 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.380185Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.15473531513134593933","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"56","size":3012283,"eTag":"04e2b00a44735a8d1c7f27d83c0d87c0","versionId":"","sequencer":"C8C7F8641D9CE117","metadata":[],"tags":[]}},"eventId":"1694025672.400661.04e2b00a44735a8d1c7f27d83c0d87c0","opaqueData":""}]}' 2023-09-06T18:54:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.916 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.383876Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9155776262464409656","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"58","size":3009470,"eTag":"172493b9ac099ec3e97a630f160b26ab","versionId":"","sequencer":"C8C7F864E4DEF517","metadata":[],"tags":[]}},"eventId":"1694025672.401989.172493b9ac099ec3e97a630f160b26ab","opaqueData":""}]}' 2023-09-06T18:54:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.917 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.391228Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.16694662071553217067","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"54","size":3010997,"eTag":"0072ca540ba470198a5568e60d786661","versionId":"","sequencer":"C8C7F86410C66A18","metadata":[],"tags":[]}},"eventId":"1694025672.409650.0072ca540ba470198a5568e60d786661","opaqueData":""}]}' 2023-09-06T18:54:38.917 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.917 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.400540Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.1923561459539483354","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"57","size":3010275,"eTag":"f781bdaf137dafbc7c08676274c15a02","versionId":"","sequencer":"C8C7F864FFCDC318","metadata":[],"tags":[]}},"eventId":"1694025672.415485.f781bdaf137dafbc7c08676274c15a02","opaqueData":""}]}' 2023-09-06T18:54:38.917 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/85 2023-09-06T18:54:38.917 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/85 2023-09-06T18:54:38.918 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.918 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/85 2023-09-06T18:54:38.918 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.918 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'YDCq1EcJ1shxd4KL33U9ZA==', 'Content-Length': '3011189', 'Expect': '100-Continue'} 2023-09-06T18:54:38.919 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.919 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.919 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.919 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.919 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.919 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.920 INFO:teuthology.orchestra.run.smithi099.stderr:YDCq1EcJ1shxd4KL33U9ZA== 2023-09-06T18:54:38.920 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.920 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.920 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/85 2023-09-06T18:54:38.920 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.921 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:H//lau2lDgmRT3ZT1hifxD4FDCA= 2023-09-06T18:54:38.921 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'YDCq1EcJ1shxd4KL33U9ZA==', 'Content-Length': '3011189', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:H//lau2lDgmRT3ZT1hifxD4FDCA='} 2023-09-06T18:54:38.921 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/86 2023-09-06T18:54:38.921 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/86 2023-09-06T18:54:38.921 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.922 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/86 2023-09-06T18:54:38.922 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.922 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'bzcWCe4Zs+Wz9mmNWMtfMA==', 'Content-Length': '3011876', 'Expect': '100-Continue'} 2023-09-06T18:54:38.922 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.922 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.923 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.923 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.923 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.923 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.923 INFO:teuthology.orchestra.run.smithi099.stderr:bzcWCe4Zs+Wz9mmNWMtfMA== 2023-09-06T18:54:38.924 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.924 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.924 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/86 2023-09-06T18:54:38.924 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.924 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:jytVWq34l+SSBNBapWZMC66VzHY= 2023-09-06T18:54:38.924 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'bzcWCe4Zs+Wz9mmNWMtfMA==', 'Content-Length': '3011876', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:jytVWq34l+SSBNBapWZMC66VzHY='} 2023-09-06T18:54:38.925 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/90 2023-09-06T18:54:38.925 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/90 2023-09-06T18:54:38.925 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.925 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/90 2023-09-06T18:54:38.925 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.926 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '6Mq+4Y9CKsucB1afANko4g==', 'Content-Length': '3013824', 'Expect': '100-Continue'} 2023-09-06T18:54:38.926 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.926 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.926 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.926 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.927 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.927 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.927 INFO:teuthology.orchestra.run.smithi099.stderr:6Mq+4Y9CKsucB1afANko4g== 2023-09-06T18:54:38.927 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.927 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.928 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/90 2023-09-06T18:54:38.928 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.928 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1hcMZJN2zAi4C/ciWwcDP+nO928= 2023-09-06T18:54:38.928 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '6Mq+4Y9CKsucB1afANko4g==', 'Content-Length': '3013824', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1hcMZJN2zAi4C/ciWwcDP+nO928='} 2023-09-06T18:54:38.928 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/87 2023-09-06T18:54:38.928 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/87 2023-09-06T18:54:38.929 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.929 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/87 2023-09-06T18:54:38.929 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.929 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'vUxwqUKsZaB1t8UY3dtHOg==', 'Content-Length': '3009715', 'Expect': '100-Continue'} 2023-09-06T18:54:38.929 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.930 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.930 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.930 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.930 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:vUxwqUKsZaB1t8UY3dtHOg== 2023-09-06T18:54:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/87 2023-09-06T18:54:38.932 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.932 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:EBRZ6AH9z0Bo+QZREF5CA5CZ9XA= 2023-09-06T18:54:38.932 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'vUxwqUKsZaB1t8UY3dtHOg==', 'Content-Length': '3009715', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:EBRZ6AH9z0Bo+QZREF5CA5CZ9XA='} 2023-09-06T18:54:38.932 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/88 2023-09-06T18:54:38.933 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/88 2023-09-06T18:54:38.933 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.933 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/88 2023-09-06T18:54:38.933 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.933 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MwPOSAa17PNqNyNds46rPQ==', 'Content-Length': '3010515', 'Expect': '100-Continue'} 2023-09-06T18:54:38.933 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.934 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.934 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.934 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.934 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:MwPOSAa17PNqNyNds46rPQ== 2023-09-06T18:54:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/88 2023-09-06T18:54:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:PYSmmu8GGwgwE32n73PXNZEqkko= 2023-09-06T18:54:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MwPOSAa17PNqNyNds46rPQ==', 'Content-Length': '3010515', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:PYSmmu8GGwgwE32n73PXNZEqkko='} 2023-09-06T18:54:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/91 2023-09-06T18:54:38.937 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/91 2023-09-06T18:54:38.937 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.937 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/91 2023-09-06T18:54:38.937 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.937 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ZTBtXbDjEjBhg8rpL9GNdw==', 'Content-Length': '3011579', 'Expect': '100-Continue'} 2023-09-06T18:54:38.938 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.938 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.938 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.938 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.938 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.939 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.939 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.939 INFO:teuthology.orchestra.run.smithi099.stderr:ZTBtXbDjEjBhg8rpL9GNdw== 2023-09-06T18:54:38.939 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.939 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.940 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/91 2023-09-06T18:54:38.940 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.940 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:nQP1DEadb8eksNgyNVzpg8Nd0as= 2023-09-06T18:54:38.940 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ZTBtXbDjEjBhg8rpL9GNdw==', 'Content-Length': '3011579', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:nQP1DEadb8eksNgyNVzpg8Nd0as='} 2023-09-06T18:54:38.940 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/89 2023-09-06T18:54:38.941 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/89 2023-09-06T18:54:38.941 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.941 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/89 2023-09-06T18:54:38.941 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.941 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '4lhMsR4INMw5LUlW+fjijg==', 'Content-Length': '3008833', 'Expect': '100-Continue'} 2023-09-06T18:54:38.942 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.942 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.942 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.942 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.942 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.943 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.943 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.943 INFO:teuthology.orchestra.run.smithi099.stderr:4lhMsR4INMw5LUlW+fjijg== 2023-09-06T18:54:38.943 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.943 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.944 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/89 2023-09-06T18:54:38.944 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.944 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:vnxVSmpqWU4SKgYVVv2CZeWLgTU= 2023-09-06T18:54:38.944 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '4lhMsR4INMw5LUlW+fjijg==', 'Content-Length': '3008833', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:vnxVSmpqWU4SKgYVVv2CZeWLgTU='} 2023-09-06T18:54:38.944 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/92 2023-09-06T18:54:38.945 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/92 2023-09-06T18:54:38.945 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.945 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/92 2023-09-06T18:54:38.945 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.945 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'd1WW15syDrKs3UdrMlkfWg==', 'Content-Length': '3010384', 'Expect': '100-Continue'} 2023-09-06T18:54:38.946 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.946 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.946 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.946 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.946 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.947 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.947 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.947 INFO:teuthology.orchestra.run.smithi099.stderr:d1WW15syDrKs3UdrMlkfWg== 2023-09-06T18:54:38.947 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.947 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.948 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/92 2023-09-06T18:54:38.948 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.948 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:4bsf40rLqAbfE0v5i1qQUo03ox4= 2023-09-06T18:54:38.948 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'd1WW15syDrKs3UdrMlkfWg==', 'Content-Length': '3010384', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:4bsf40rLqAbfE0v5i1qQUo03ox4='} 2023-09-06T18:54:38.948 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/93 2023-09-06T18:54:38.949 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/93 2023-09-06T18:54:38.949 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.949 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/93 2023-09-06T18:54:38.949 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.949 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ApVRZRTGuJLDF4R7yT+TnA==', 'Content-Length': '3008841', 'Expect': '100-Continue'} 2023-09-06T18:54:38.949 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.950 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.950 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.950 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.950 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.950 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.951 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.951 INFO:teuthology.orchestra.run.smithi099.stderr:ApVRZRTGuJLDF4R7yT+TnA== 2023-09-06T18:54:38.951 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.951 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.951 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/93 2023-09-06T18:54:38.952 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.952 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:T0yfDwfwBTB5XlDGzeKSZspocV0= 2023-09-06T18:54:38.952 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ApVRZRTGuJLDF4R7yT+TnA==', 'Content-Length': '3008841', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:T0yfDwfwBTB5XlDGzeKSZspocV0='} 2023-09-06T18:54:38.952 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/95 2023-09-06T18:54:38.952 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/95 2023-09-06T18:54:38.953 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.953 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/95 2023-09-06T18:54:38.953 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.953 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'IlteDkzUvp19xkYkWXYCEw==', 'Content-Length': '3010755', 'Expect': '100-Continue'} 2023-09-06T18:54:38.953 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.954 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.954 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.954 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.954 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.954 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.955 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.955 INFO:teuthology.orchestra.run.smithi099.stderr:IlteDkzUvp19xkYkWXYCEw== 2023-09-06T18:54:38.955 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.955 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.955 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/95 2023-09-06T18:54:38.956 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.956 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:qU8kqZa6Kd3gi2mTN1yByHiSPhs= 2023-09-06T18:54:38.956 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'IlteDkzUvp19xkYkWXYCEw==', 'Content-Length': '3010755', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:qU8kqZa6Kd3gi2mTN1yByHiSPhs='} 2023-09-06T18:54:38.956 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/94 2023-09-06T18:54:38.956 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/94 2023-09-06T18:54:38.956 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.957 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/94 2023-09-06T18:54:38.957 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.957 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GJkR3zTdK3YwvIGvfcQsJA==', 'Content-Length': '3011083', 'Expect': '100-Continue'} 2023-09-06T18:54:38.957 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.957 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.958 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.958 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.958 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.958 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.958 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.959 INFO:teuthology.orchestra.run.smithi099.stderr:GJkR3zTdK3YwvIGvfcQsJA== 2023-09-06T18:54:38.959 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.959 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.959 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/94 2023-09-06T18:54:38.959 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.960 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1TCHMs8abxmMC5byV+PH4dfiIn8= 2023-09-06T18:54:38.960 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GJkR3zTdK3YwvIGvfcQsJA==', 'Content-Length': '3011083', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1TCHMs8abxmMC5byV+PH4dfiIn8='} 2023-09-06T18:54:38.960 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/96 2023-09-06T18:54:38.960 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/96 2023-09-06T18:54:38.960 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.961 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/96 2023-09-06T18:54:38.961 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.961 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'OkRpNuHoekxqIV5d/AEmQw==', 'Content-Length': '3010153', 'Expect': '100-Continue'} 2023-09-06T18:54:38.961 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.961 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.961 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.962 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.962 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.962 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.962 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.962 INFO:teuthology.orchestra.run.smithi099.stderr:OkRpNuHoekxqIV5d/AEmQw== 2023-09-06T18:54:38.963 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.963 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.963 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/96 2023-09-06T18:54:38.963 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.963 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VNYADlT08pfUnJaj4kH1qEp0w0M= 2023-09-06T18:54:38.964 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'OkRpNuHoekxqIV5d/AEmQw==', 'Content-Length': '3010153', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VNYADlT08pfUnJaj4kH1qEp0w0M='} 2023-09-06T18:54:38.964 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/97 2023-09-06T18:54:38.964 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/97 2023-09-06T18:54:38.964 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.964 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/97 2023-09-06T18:54:38.965 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.965 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1Wi/eLV38vm0BZtxAI2RNA==', 'Content-Length': '3011110', 'Expect': '100-Continue'} 2023-09-06T18:54:38.965 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.965 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.965 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.966 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.966 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.966 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.966 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.966 INFO:teuthology.orchestra.run.smithi099.stderr:1Wi/eLV38vm0BZtxAI2RNA== 2023-09-06T18:54:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/97 2023-09-06T18:54:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:dNHETCwYqsClR6tlXTHGA3vYsm8= 2023-09-06T18:54:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1Wi/eLV38vm0BZtxAI2RNA==', 'Content-Length': '3011110', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:dNHETCwYqsClR6tlXTHGA3vYsm8='} 2023-09-06T18:54:38.968 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"1fb90959c654fe785879531b45801165"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d10dc894fba3a0fe-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.968 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"d89f35dcd16238ee467a5428076e6fb2"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000002948bc6c2773f2c-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.968 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"d5ad402743f8466d206045f34667c425"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000884955346bced603-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.968 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"292f59c359f8532ff8cb3c6d6b137aee"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000673b7d81c86d3e10-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.969 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.969 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.969 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.271245Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11932110549230009864","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"51","size":3009659,"eTag":"0cccd9d38d2db2c71516d96385b879ab","versionId":"","sequencer":"C8C7F8648247D918","metadata":[],"tags":[]}},"eventId":"1694025672.416892.0cccd9d38d2db2c71516d96385b879ab","opaqueData":""}]}' 2023-09-06T18:54:38.969 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.969 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.232206Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.2034980962141960969","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"48","size":3010416,"eTag":"185b624ab367f0b79f6c2a2b39f86dc0","versionId":"","sequencer":"C8C7F8645449D918","metadata":[],"tags":[]}},"eventId":"1694025672.416893.185b624ab367f0b79f6c2a2b39f86dc0","opaqueData":""}]}' 2023-09-06T18:54:38.969 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.970 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.275918Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.7174042032887746701","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"52","size":3011074,"eTag":"3f35a2ad28d76a26eace26fb31b17a53","versionId":"","sequencer":"C8C7F864EE77D918","metadata":[],"tags":[]}},"eventId":"1694025672.416905.3f35a2ad28d76a26eace26fb31b17a53","opaqueData":""}]}' 2023-09-06T18:54:38.970 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.970 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.211710Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.6630435446289497429","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"49","size":3009028,"eTag":"50e4e8c49dce88e415b28cf161f8c579","versionId":"","sequencer":"C8C7F8642842D918","metadata":[],"tags":[]}},"eventId":"1694025672.416891.50e4e8c49dce88e415b28cf161f8c579","opaqueData":""}]}' 2023-09-06T18:54:38.970 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.970 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.971 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.421172Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11466350703732751786","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"60","size":3011279,"eTag":"168c2a30c724cf4556521b9a3717b69d","versionId":"","sequencer":"C8C7F8644797DB19","metadata":[],"tags":[]}},"eventId":"1694025672.433821.168c2a30c724cf4556521b9a3717b69d","opaqueData":""}]}' 2023-09-06T18:54:38.971 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.971 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.418264Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.8300753514046053426","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"59","size":3013085,"eTag":"772f1465f62755b781baed1e10410077","versionId":"","sequencer":"C8C7F864D000E319","metadata":[],"tags":[]}},"eventId":"1694025672.434307.772f1465f62755b781baed1e10410077","opaqueData":""}]}' 2023-09-06T18:54:38.971 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.971 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.441631Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11756071881812628015","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"62","size":3012829,"eTag":"2fe0a76ed44122c99c2a359ad5b82a01","versionId":"","sequencer":"C8C7F864B3056E1B","metadata":[],"tags":[]}},"eventId":"1694025672.460195.2fe0a76ed44122c99c2a359ad5b82a01","opaqueData":""}]}' 2023-09-06T18:54:38.972 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.972 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.446305Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.1933671264602696900","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"64","size":3012286,"eTag":"b13198e77f216282d35dfa6a4e523ae9","versionId":"","sequencer":"C8C7F8646EE8801B","metadata":[],"tags":[]}},"eventId":"1694025672.461432.b13198e77f216282d35dfa6a4e523ae9","opaqueData":""}]}' 2023-09-06T18:54:38.972 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.972 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.465297Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.8331992543855121239","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"63","size":3010173,"eTag":"4f98bbeba655249dde5d7394b1cea514","versionId":"","sequencer":"C8C7F864AC5D861C","metadata":[],"tags":[]}},"eventId":"1694025672.478567.4f98bbeba655249dde5d7394b1cea514","opaqueData":""}]}' 2023-09-06T18:54:38.972 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.973 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.471542Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.15250865929328720718","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"65","size":3010799,"eTag":"258e3e79c6f6126c09bfffd17f7a0287","versionId":"","sequencer":"C8C7F864FD890F1D","metadata":[],"tags":[]}},"eventId":"1694025672.487557.258e3e79c6f6126c09bfffd17f7a0287","opaqueData":""}]}' 2023-09-06T18:54:38.973 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.973 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.489607Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.14600939339801516458","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"67","size":3011407,"eTag":"084c5cb41c52bc97d718ab3f2bf4fd31","versionId":"","sequencer":"C8C7F864E20DC71D","metadata":[],"tags":[]}},"eventId":"1694025672.499584.084c5cb41c52bc97d718ab3f2bf4fd31","opaqueData":""}]}' 2023-09-06T18:54:38.973 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.973 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.499150Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.3141063678502515518","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"70","size":3011260,"eTag":"0028c5c07e4d1a528474f5850ecd14ba","versionId":"","sequencer":"C8C7F864EDBC901E","metadata":[],"tags":[]}},"eventId":"1694025672.512802.0028c5c07e4d1a528474f5850ecd14ba","opaqueData":""}]}' 2023-09-06T18:54:38.974 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"f6ac05a5e7c419fdde31a79b438952bb"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ffcd406ef951459b-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.974 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"e2082ae21e6cf279dd0cfa72aa4c63e0"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004c2d0aff91a74116-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.974 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/98 2023-09-06T18:54:38.974 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/98 2023-09-06T18:54:38.974 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.975 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/98 2023-09-06T18:54:38.975 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.975 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FruUTzNJSEDHJpOgQE21Q==', 'Content-Length': '3011202', 'Expect': '100-Continue'} 2023-09-06T18:54:38.975 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.975 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.976 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.976 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.976 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.976 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.976 INFO:teuthology.orchestra.run.smithi099.stderr:3FruUTzNJSEDHJpOgQE21Q== 2023-09-06T18:54:38.977 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.977 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.977 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/98 2023-09-06T18:54:38.977 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.977 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:mNSz2fBFJ+HQyMBbfHNdv778AeE= 2023-09-06T18:54:38.978 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FruUTzNJSEDHJpOgQE21Q==', 'Content-Length': '3011202', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:mNSz2fBFJ+HQyMBbfHNdv778AeE='} 2023-09-06T18:54:38.978 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/99 2023-09-06T18:54:38.978 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/99 2023-09-06T18:54:38.978 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:38.978 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/99 2023-09-06T18:54:38.979 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:38.979 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hJWNhlOi09qLpvsl0uu7Cg==', 'Content-Length': '3010645', 'Expect': '100-Continue'} 2023-09-06T18:54:38.979 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:38.979 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:38.979 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:38.980 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:38.980 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:38.980 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:38.980 INFO:teuthology.orchestra.run.smithi099.stderr:hJWNhlOi09qLpvsl0uu7Cg== 2023-09-06T18:54:38.980 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:38.980 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:38.981 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/99 2023-09-06T18:54:38.981 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:38.981 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Atvf3JbMyzqzFA+QlmjLbhjVYnk= 2023-09-06T18:54:38.981 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hJWNhlOi09qLpvsl0uu7Cg==', 'Content-Length': '3010645', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Atvf3JbMyzqzFA+QlmjLbhjVYnk='} 2023-09-06T18:54:38.981 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"0cccd9d38d2db2c71516d96385b879ab"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a5975f53538bbe08-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.982 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.982 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.982 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.308230Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.15038305735325146487","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"53","size":3012434,"eTag":"19e2e290cacec3bdf74cd189efcd00c7","versionId":"","sequencer":"C8C7F864BB41D21E","metadata":[],"tags":[]}},"eventId":"1694025672.517095.19e2e290cacec3bdf74cd189efcd00c7","opaqueData":""}]}' 2023-09-06T18:54:38.982 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.982 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.336382Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.8596261825739537400","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"55","size":3007528,"eTag":"1c2533bdb939112a19b882ab67e59b87","versionId":"","sequencer":"C8C7F8640EA7D21E","metadata":[],"tags":[]}},"eventId":"1694025672.517121.1c2533bdb939112a19b882ab67e59b87","opaqueData":""}]}' 2023-09-06T18:54:38.983 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.983 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.383876Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.9155776262464409656","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"58","size":3009470,"eTag":"172493b9ac099ec3e97a630f160b26ab","versionId":"","sequencer":"C8C7F8645EBED31E","metadata":[],"tags":[]}},"eventId":"1694025672.517193.172493b9ac099ec3e97a630f160b26ab","opaqueData":""}]}' 2023-09-06T18:54:38.983 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.983 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.368387Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.1696421320166965460","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"61","size":3010350,"eTag":"9cba81f887663b7b2a822e6dab4bb3d6","versionId":"","sequencer":"C8C7F86408ABD31E","metadata":[],"tags":[]}},"eventId":"1694025672.517188.9cba81f887663b7b2a822e6dab4bb3d6","opaqueData":""}]}' 2023-09-06T18:54:38.983 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.984 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.400540Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.1923561459539483354","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"57","size":3010275,"eTag":"f781bdaf137dafbc7c08676274c15a02","versionId":"","sequencer":"C8C7F86429FCD31E","metadata":[],"tags":[]}},"eventId":"1694025672.517209.f781bdaf137dafbc7c08676274c15a02","opaqueData":""}]}' 2023-09-06T18:54:38.984 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.984 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.391228Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.16694662071553217067","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"54","size":3010997,"eTag":"0072ca540ba470198a5568e60d786661","versionId":"","sequencer":"C8C7F86442F9D31E","metadata":[],"tags":[]}},"eventId":"1694025672.517208.0072ca540ba470198a5568e60d786661","opaqueData":""}]}' 2023-09-06T18:54:38.984 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_2: 2023-09-06T18:54:38.985 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.380185Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.15473531513134593933","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_2","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"56","size":3012283,"eTag":"04e2b00a44735a8d1c7f27d83c0d87c0","versionId":"","sequencer":"C8C7F86404C2D31E","metadata":[],"tags":[]}},"eventId":"1694025672.517194.04e2b00a44735a8d1c7f27d83c0d87c0","opaqueData":""}]}' 2023-09-06T18:54:38.985 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"50e4e8c49dce88e415b28cf161f8c579"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005c0409e8aa3e7d55-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.985 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"1c2533bdb939112a19b882ab67e59b87"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000774c0e95e774bbf8-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.985 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"f781bdaf137dafbc7c08676274c15a02"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001ab1dcf4bb7b9ada-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.985 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"0072ca540ba470198a5568e60d786661"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e7af5ab3b41dd22b-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.986 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"04e2b00a44735a8d1c7f27d83c0d87c0"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d6bd06edcd33378d-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.986 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"172493b9ac099ec3e97a630f160b26ab"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007f0fda0057251038-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.986 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"9cba81f887663b7b2a822e6dab4bb3d6"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000178ae633e1f380d4-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.986 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"185b624ab367f0b79f6c2a2b39f86dc0"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001c3db4673d277b09-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.986 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"3f35a2ad28d76a26eace26fb31b17a53"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000638f51381dfed08d-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.987 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"19e2e290cacec3bdf74cd189efcd00c7"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d0b2cb63d1c65d77-0064f8c7c7-11b2-default'), ('Server', 'Ceph Object Gateway (reef)'), ('Date', 'Wed, 06 Sep 2023 18:41:12 GMT'), ('Connection', 'Keep-Alive')] 2023-09-06T18:54:38.987 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:38.987 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.987 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.507087Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.14939323112226896726","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"68","size":3009821,"eTag":"0ba0fea31badea4af0bd43e7f44e9aa9","versionId":"","sequencer":"C8C7F8644736DF1E","metadata":[],"tags":[]}},"eventId":"1694025672.517944.0ba0fea31badea4af0bd43e7f44e9aa9","opaqueData":""}]}' 2023-09-06T18:54:38.987 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.988 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.516853Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.17036821738190330741","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"66","size":3010803,"eTag":"1ffecf55168185a8938011b0b28700f6","versionId":"","sequencer":"C8C7F864B06A901F","metadata":[],"tags":[]}},"eventId":"1694025672.529558.1ffecf55168185a8938011b0b28700f6","opaqueData":""}]}' 2023-09-06T18:54:38.988 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.988 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.520972Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.13224037536041990546","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"69","size":3009018,"eTag":"4e5a0b07d94f041dbf4ed941d86cb5c0","versionId":"","sequencer":"C8C7F864B19FF31F","metadata":[],"tags":[]}},"eventId":"1694025672.536059.4e5a0b07d94f041dbf4ed941d86cb5c0","opaqueData":""}]}' 2023-09-06T18:54:38.988 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.988 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.529057Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.13512759930988613328","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"71","size":3010257,"eTag":"a2707d7695ee203e4b7a1f6273aa5eae","versionId":"","sequencer":"C8C7F864CAC43D20","metadata":[],"tags":[]}},"eventId":"1694025672.540918.a2707d7695ee203e4b7a1f6273aa5eae","opaqueData":""}]}' 2023-09-06T18:54:38.989 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.989 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.559515Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.11668632919765137588","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"73","size":3008097,"eTag":"8afc19ed5761ff901ecf3b2982d57190","versionId":"","sequencer":"C8C7F864073BBE22","metadata":[],"tags":[]}},"eventId":"1694025672.582892.8afc19ed5761ff901ecf3b2982d57190","opaqueData":""}]}' 2023-09-06T18:54:38.989 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.989 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.554817Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.15012134143007821205","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"72","size":3012528,"eTag":"e7cfecc8dbe7304cac21c1d201df0d06","versionId":"","sequencer":"C8C7F86442DFC722","metadata":[],"tags":[]}},"eventId":"1694025672.583524.e7cfecc8dbe7304cac21c1d201df0d06","opaqueData":""}]}' 2023-09-06T18:54:38.989 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.990 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.564058Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.7627115772863187657","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"74","size":3009021,"eTag":"c07e0ee06d1c51add15594d92fc071f2","versionId":"","sequencer":"C8C7F8649E9F0823","metadata":[],"tags":[]}},"eventId":"1694025672.587767.c07e0ee06d1c51add15594d92fc071f2","opaqueData":""}]}' 2023-09-06T18:54:38.990 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.990 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.573488Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.15346016685763430081","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"75","size":3010936,"eTag":"d7728fa098ec59999be3c160d0ae91fe","versionId":"","sequencer":"C8C7F864BCE41E23","metadata":[],"tags":[]}},"eventId":"1694025672.589227.d7728fa098ec59999be3c160d0ae91fe","opaqueData":""}]}' 2023-09-06T18:54:38.990 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.991 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.580609Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.6159278212740862476","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"78","size":3012430,"eTag":"2c89c323fe2ceefb429a651d97619981","versionId":"","sequencer":"C8C7F864EFE3BA23","metadata":[],"tags":[]}},"eventId":"1694025672.599450.2c89c323fe2ceefb429a651d97619981","opaqueData":""}]}' 2023-09-06T18:54:38.991 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.991 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.571736Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.10788747510620649948","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"76","size":3012500,"eTag":"3b9ab90d4be90e37935b1d94412d1bff","versionId":"","sequencer":"C8C7F864A206E223","metadata":[],"tags":[]}},"eventId":"1694025672.602015.3b9ab90d4be90e37935b1d94412d1bff","opaqueData":""}]}' 2023-09-06T18:54:38.991 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.991 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.587472Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.12953615356119277615","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"79","size":3009702,"eTag":"6b3f26de02f0037adfb8eb86a0852522","versionId":"","sequencer":"C8C7F864C2414224","metadata":[],"tags":[]}},"eventId":"1694025672.608321.6b3f26de02f0037adfb8eb86a0852522","opaqueData":""}]}' 2023-09-06T18:54:38.992 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.992 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.585165Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.12087616488069914503","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"77","size":3008801,"eTag":"f01ddc73fb7c4e94591797027abe9d11","versionId":"","sequencer":"C8C7F8647B264124","metadata":[],"tags":[]}},"eventId":"1694025672.608249.f01ddc73fb7c4e94591797027abe9d11","opaqueData":""}]}' 2023-09-06T18:54:38.992 INFO:teuthology.orchestra.run.smithi099.stderr:bucket_notification.test_bn: INFO: AMQP received event for topic cjhpse-2_topic_1: 2023-09-06T18:54:38.992 INFO:teuthology.orchestra.run.smithi099.stderr: b'{"Records":[{"eventVersion":"2.2","eventSource":"ceph:s3","awsRegion":"default","eventTime":"2023-09-06T18:41:12.587020Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"foo.client.0"},"requestParameters":{"sourceIPAddress":""},"responseElements":{"x-amz-request-id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4530.13753006667209164815","x-amz-id-2":"11b2-default-default"},"s3":{"s3SchemaVersion":"1.0","configurationId":"cjhpse-2_notif_1","bucket":{"name":"cjhpse-2","ownerIdentity":{"principalId":"foo.client.0"},"arn":"arn:aws:s3:default::cjhpse-2","id":"d19ec83f-a74c-4fd7-8a2d-747094fce3d9.4463.2"},"object":{"key":"81","size":3011797,"eTag":"4e9c597a098dd80fac277b07b4d477e2","versionId":"","sequencer":"C8C7F86440616F24","metadata":[],"tags":[]}},"eventId":"1694025672.611279.4e9c597a098dd80fac277b07b4d477e2","opaqueData":""}]}' 2023-09-06T18:54:38.992 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.993 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.993 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.993 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.993 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.993 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.994 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.994 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.994 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.994 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.995 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.995 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.995 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.995 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.995 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.995 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.996 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.996 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.996 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.996 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.996 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.997 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.997 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.997 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.997 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.997 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.998 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.998 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.998 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.998 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.998 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.999 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.999 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.999 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:38.999 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:38.999 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.000 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.000 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.000 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.000 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.000 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.001 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.001 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.001 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.001 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.001 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.002 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.002 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.002 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.002 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.002 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.003 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.003 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.003 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.003 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.003 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.003 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.004 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.004 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.004 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.004 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.005 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.005 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.005 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.005 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.005 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.006 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.006 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.006 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.006 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.006 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.006 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.007 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.007 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.007 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.007 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.007 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.007 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.008 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered RemoteDisconnected exception, reconnecting 2023-09-06T18:54:39.008 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.008 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.008 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.008 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.009 INFO:teuthology.orchestra.run.smithi099.stderr:H/7PVRaBhaiTgBGwsocA9g== 2023-09-06T18:54:39.009 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.009 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.009 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/66 2023-09-06T18:54:39.009 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.009 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:36FUK1BTY8M1HxewVhurZeS63eU= 2023-09-06T18:54:39.010 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'H/7PVRaBhaiTgBGwsocA9g==', 'Content-Length': '3010803', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:36FUK1BTY8M1HxewVhurZeS63eU='} 2023-09-06T18:54:39.010 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.010 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.010 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.010 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.011 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.011 INFO:teuthology.orchestra.run.smithi099.stderr:8B3cc/t8TpRZF5cCer6dEQ== 2023-09-06T18:54:39.011 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.011 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.011 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/77 2023-09-06T18:54:39.011 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.012 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VWVCrMVG65GwMoPOxodu30Gx5IY= 2023-09-06T18:54:39.012 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '8B3cc/t8TpRZF5cCer6dEQ==', 'Content-Length': '3008801', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VWVCrMVG65GwMoPOxodu30Gx5IY='} 2023-09-06T18:54:39.012 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.012 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.012 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.013 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.013 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.013 INFO:teuthology.orchestra.run.smithi099.stderr:SCQrX/nmwhId+jfW+hbFKA== 2023-09-06T18:54:39.013 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.013 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.013 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/82 2023-09-06T18:54:39.014 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.014 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:0g54EVFLjN5wdXtHGIks2sEB4/o= 2023-09-06T18:54:39.014 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'SCQrX/nmwhId+jfW+hbFKA==', 'Content-Length': '3013892', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:0g54EVFLjN5wdXtHGIks2sEB4/o='} 2023-09-06T18:54:39.014 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.014 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.015 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.015 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.015 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.015 INFO:teuthology.orchestra.run.smithi099.stderr:6Mq+4Y9CKsucB1afANko4g== 2023-09-06T18:54:39.015 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.016 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.016 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/90 2023-09-06T18:54:39.016 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.016 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1hcMZJN2zAi4C/ciWwcDP+nO928= 2023-09-06T18:54:39.016 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '6Mq+4Y9CKsucB1afANko4g==', 'Content-Length': '3013824', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1hcMZJN2zAi4C/ciWwcDP+nO928='} 2023-09-06T18:54:39.017 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.017 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.017 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.017 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.017 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.018 INFO:teuthology.orchestra.run.smithi099.stderr:OkRpNuHoekxqIV5d/AEmQw== 2023-09-06T18:54:39.018 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.018 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.018 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/96 2023-09-06T18:54:39.018 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.019 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VNYADlT08pfUnJaj4kH1qEp0w0M= 2023-09-06T18:54:39.019 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'OkRpNuHoekxqIV5d/AEmQw==', 'Content-Length': '3010153', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VNYADlT08pfUnJaj4kH1qEp0w0M='} 2023-09-06T18:54:39.019 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.019 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.019 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.019 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.020 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.020 INFO:teuthology.orchestra.run.smithi099.stderr:ZTBtXbDjEjBhg8rpL9GNdw== 2023-09-06T18:54:39.020 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.020 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.020 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/91 2023-09-06T18:54:39.021 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.021 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:nQP1DEadb8eksNgyNVzpg8Nd0as= 2023-09-06T18:54:39.021 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ZTBtXbDjEjBhg8rpL9GNdw==', 'Content-Length': '3011579', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:nQP1DEadb8eksNgyNVzpg8Nd0as='} 2023-09-06T18:54:39.021 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.022 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.022 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.022 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.022 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.022 INFO:teuthology.orchestra.run.smithi099.stderr:CExctBxSvJfXGKs/K/T9MQ== 2023-09-06T18:54:39.023 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.023 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.023 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/67 2023-09-06T18:54:39.023 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.023 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Kg+8r2tVv0ClHQIwVtGnpmy4f0A= 2023-09-06T18:54:39.024 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'CExctBxSvJfXGKs/K/T9MQ==', 'Content-Length': '3011407', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Kg+8r2tVv0ClHQIwVtGnpmy4f0A='} 2023-09-06T18:54:39.024 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.024 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.024 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.024 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.025 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.025 INFO:teuthology.orchestra.run.smithi099.stderr:C6D+oxut6krwvUPn9E6aqQ== 2023-09-06T18:54:39.025 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.025 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.025 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/68 2023-09-06T18:54:39.026 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.026 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:CtTT22ewDAnC9YahPSmJI1/2Xt0= 2023-09-06T18:54:39.026 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C6D+oxut6krwvUPn9E6aqQ==', 'Content-Length': '3009821', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:CtTT22ewDAnC9YahPSmJI1/2Xt0='} 2023-09-06T18:54:39.026 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.026 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.027 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.027 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.027 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.027 INFO:teuthology.orchestra.run.smithi099.stderr:MwPOSAa17PNqNyNds46rPQ== 2023-09-06T18:54:39.027 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.028 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.028 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/88 2023-09-06T18:54:39.028 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.028 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:PYSmmu8GGwgwE32n73PXNZEqkko= 2023-09-06T18:54:39.028 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MwPOSAa17PNqNyNds46rPQ==', 'Content-Length': '3010515', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:PYSmmu8GGwgwE32n73PXNZEqkko='} 2023-09-06T18:54:39.028 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.029 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.029 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.029 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.029 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.029 INFO:teuthology.orchestra.run.smithi099.stderr:az8m3gLwA3rfuOuGoIUlIg== 2023-09-06T18:54:39.030 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.030 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.030 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/79 2023-09-06T18:54:39.030 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.030 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:g3Ny3rsaOzsW2WwvyD68FiQb99s= 2023-09-06T18:54:39.031 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'az8m3gLwA3rfuOuGoIUlIg==', 'Content-Length': '3009702', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:g3Ny3rsaOzsW2WwvyD68FiQb99s='} 2023-09-06T18:54:39.031 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.031 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.031 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.031 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.031 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.032 INFO:teuthology.orchestra.run.smithi099.stderr:ACjFwH5NGlKEdPWFDs0Uug== 2023-09-06T18:54:39.032 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.032 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.032 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/70 2023-09-06T18:54:39.032 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.033 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Gvl3cUJ2iRK2IcsrZZrixV76LeQ= 2023-09-06T18:54:39.033 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ACjFwH5NGlKEdPWFDs0Uug==', 'Content-Length': '3011260', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Gvl3cUJ2iRK2IcsrZZrixV76LeQ='} 2023-09-06T18:54:39.033 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.033 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.033 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.033 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.034 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.034 INFO:teuthology.orchestra.run.smithi099.stderr:C/iQp/RnJcz00qfhYKVejg== 2023-09-06T18:54:39.034 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.034 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.034 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/83 2023-09-06T18:54:39.035 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.035 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VBQiPvpgkgY0eGav2GjAQBMnjks= 2023-09-06T18:54:39.035 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C/iQp/RnJcz00qfhYKVejg==', 'Content-Length': '3008624', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VBQiPvpgkgY0eGav2GjAQBMnjks='} 2023-09-06T18:54:39.035 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.035 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.035 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.036 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.036 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.036 INFO:teuthology.orchestra.run.smithi099.stderr:MwPOSAa17PNqNyNds46rPQ== 2023-09-06T18:54:39.036 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.036 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.037 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/88 2023-09-06T18:54:39.037 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.037 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:PYSmmu8GGwgwE32n73PXNZEqkko= 2023-09-06T18:54:39.037 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MwPOSAa17PNqNyNds46rPQ==', 'Content-Length': '3010515', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:PYSmmu8GGwgwE32n73PXNZEqkko='} 2023-09-06T18:54:39.037 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.038 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.038 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.038 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.038 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.038 INFO:teuthology.orchestra.run.smithi099.stderr:4lhMsR4INMw5LUlW+fjijg== 2023-09-06T18:54:39.039 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.039 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.039 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/89 2023-09-06T18:54:39.039 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.039 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:vnxVSmpqWU4SKgYVVv2CZeWLgTU= 2023-09-06T18:54:39.040 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '4lhMsR4INMw5LUlW+fjijg==', 'Content-Length': '3008833', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:vnxVSmpqWU4SKgYVVv2CZeWLgTU='} 2023-09-06T18:54:39.040 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.040 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.040 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.040 INFO:teuthology.orchestra.run.smithi099.stderr:TloLB9lPBB2/TtlB2Gy1wA== 2023-09-06T18:54:39.041 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.041 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.041 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/69 2023-09-06T18:54:39.041 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.041 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6qcEOM3G1/zk11ieYYG76Z/6oiA= 2023-09-06T18:54:39.041 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TloLB9lPBB2/TtlB2Gy1wA==', 'Content-Length': '3009018', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6qcEOM3G1/zk11ieYYG76Z/6oiA='} 2023-09-06T18:54:39.042 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.042 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.042 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.042 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.042 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.043 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.043 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.043 INFO:teuthology.orchestra.run.smithi099.stderr:vUxwqUKsZaB1t8UY3dtHOg== 2023-09-06T18:54:39.043 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.043 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.043 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/87 2023-09-06T18:54:39.044 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.044 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:EBRZ6AH9z0Bo+QZREF5CA5CZ9XA= 2023-09-06T18:54:39.044 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'vUxwqUKsZaB1t8UY3dtHOg==', 'Content-Length': '3009715', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:EBRZ6AH9z0Bo+QZREF5CA5CZ9XA='} 2023-09-06T18:54:39.044 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.044 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.045 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.045 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.045 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.045 INFO:teuthology.orchestra.run.smithi099.stderr:IlteDkzUvp19xkYkWXYCEw== 2023-09-06T18:54:39.045 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.046 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.046 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/95 2023-09-06T18:54:39.046 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.046 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:qU8kqZa6Kd3gi2mTN1yByHiSPhs= 2023-09-06T18:54:39.046 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'IlteDkzUvp19xkYkWXYCEw==', 'Content-Length': '3010755', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:qU8kqZa6Kd3gi2mTN1yByHiSPhs='} 2023-09-06T18:54:39.047 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.047 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.047 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.047 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.047 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.048 INFO:teuthology.orchestra.run.smithi099.stderr:LInDI/4s7vtCmmUdl2GZgQ== 2023-09-06T18:54:39.048 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.048 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.048 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/78 2023-09-06T18:54:39.048 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.048 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:wTYx6jGw8NbNYPV0mbzgPssE+PE= 2023-09-06T18:54:39.049 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'LInDI/4s7vtCmmUdl2GZgQ==', 'Content-Length': '3012430', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:wTYx6jGw8NbNYPV0mbzgPssE+PE='} 2023-09-06T18:54:39.049 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.049 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.049 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.049 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.050 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.050 INFO:teuthology.orchestra.run.smithi099.stderr:wH4O4G0cUa3RVZTZL8Bx8g== 2023-09-06T18:54:39.050 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.050 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.050 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/74 2023-09-06T18:54:39.051 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.051 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:oXpzU2EWQ1vcOuvfW97TE3ZNzJg= 2023-09-06T18:54:39.051 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'wH4O4G0cUa3RVZTZL8Bx8g==', 'Content-Length': '3009021', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:oXpzU2EWQ1vcOuvfW97TE3ZNzJg='} 2023-09-06T18:54:39.051 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.051 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.051 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.052 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.052 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.052 INFO:teuthology.orchestra.run.smithi099.stderr:L+CnbtRBIsmcKjWa1bgqAQ== 2023-09-06T18:54:39.052 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.052 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.053 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/62 2023-09-06T18:54:39.053 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.053 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:YvmJ7d/T+PZLDfTssCi26WRDB6U= 2023-09-06T18:54:39.053 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'L+CnbtRBIsmcKjWa1bgqAQ==', 'Content-Length': '3012829', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:YvmJ7d/T+PZLDfTssCi26WRDB6U='} 2023-09-06T18:54:39.053 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.054 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.054 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.054 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.054 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.054 INFO:teuthology.orchestra.run.smithi099.stderr:MILNOxKsGQkBzSCgDskNTA== 2023-09-06T18:54:39.055 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.055 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.055 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/80 2023-09-06T18:54:39.055 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.055 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:5sx0j0PgApPbvZFYmcsddlB3Dig= 2023-09-06T18:54:39.056 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MILNOxKsGQkBzSCgDskNTA==', 'Content-Length': '3009370', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:5sx0j0PgApPbvZFYmcsddlB3Dig='} 2023-09-06T18:54:39.056 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.056 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.056 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.056 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.057 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.057 INFO:teuthology.orchestra.run.smithi099.stderr:ivwZ7Vdh/5AezzspgtVxkA== 2023-09-06T18:54:39.057 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.057 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.057 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/73 2023-09-06T18:54:39.057 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.058 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:rAgrUjCY/2UU1peWmmZ9tsZx824= 2023-09-06T18:54:39.058 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ivwZ7Vdh/5AezzspgtVxkA==', 'Content-Length': '3008097', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:rAgrUjCY/2UU1peWmmZ9tsZx824='} 2023-09-06T18:54:39.058 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.058 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.058 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.059 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.059 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.059 INFO:teuthology.orchestra.run.smithi099.stderr:C6D+oxut6krwvUPn9E6aqQ== 2023-09-06T18:54:39.059 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.059 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.060 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/68 2023-09-06T18:54:39.060 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.060 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:CtTT22ewDAnC9YahPSmJI1/2Xt0= 2023-09-06T18:54:39.060 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C6D+oxut6krwvUPn9E6aqQ==', 'Content-Length': '3009821', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:CtTT22ewDAnC9YahPSmJI1/2Xt0='} 2023-09-06T18:54:39.060 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.061 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.061 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.061 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.061 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.061 INFO:teuthology.orchestra.run.smithi099.stderr:hJWNhlOi09qLpvsl0uu7Cg== 2023-09-06T18:54:39.061 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.062 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.062 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/99 2023-09-06T18:54:39.062 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.062 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Atvf3JbMyzqzFA+QlmjLbhjVYnk= 2023-09-06T18:54:39.062 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hJWNhlOi09qLpvsl0uu7Cg==', 'Content-Length': '3010645', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Atvf3JbMyzqzFA+QlmjLbhjVYnk='} 2023-09-06T18:54:39.063 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.063 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.063 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.063 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.063 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.064 INFO:teuthology.orchestra.run.smithi099.stderr:hJWNhlOi09qLpvsl0uu7Cg== 2023-09-06T18:54:39.064 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.064 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.064 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/99 2023-09-06T18:54:39.064 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.064 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Atvf3JbMyzqzFA+QlmjLbhjVYnk= 2023-09-06T18:54:39.065 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hJWNhlOi09qLpvsl0uu7Cg==', 'Content-Length': '3010645', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Atvf3JbMyzqzFA+QlmjLbhjVYnk='} 2023-09-06T18:54:39.065 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.065 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.065 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.065 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.065 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.066 INFO:teuthology.orchestra.run.smithi099.stderr:TpxZegmN2A+sJ3sHtNR34g== 2023-09-06T18:54:39.066 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.066 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.066 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/81 2023-09-06T18:54:39.066 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.067 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6EPFOpoyygYmIMj+Rug1f0pP8YM= 2023-09-06T18:54:39.067 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TpxZegmN2A+sJ3sHtNR34g==', 'Content-Length': '3011797', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6EPFOpoyygYmIMj+Rug1f0pP8YM='} 2023-09-06T18:54:39.067 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.067 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.068 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.068 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.068 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.068 INFO:teuthology.orchestra.run.smithi099.stderr:T5i766ZVJJ3eXXOUsc6lFA== 2023-09-06T18:54:39.068 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.068 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.069 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/63 2023-09-06T18:54:39.069 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.069 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:RYEZ13QtbXJqx9VanMv5lwSwvYo= 2023-09-06T18:54:39.069 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'T5i766ZVJJ3eXXOUsc6lFA==', 'Content-Length': '3010173', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:RYEZ13QtbXJqx9VanMv5lwSwvYo='} 2023-09-06T18:54:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:Pynp8NrbQabMkK3jYh9YCQ== 2023-09-06T18:54:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/84 2023-09-06T18:54:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:DWb43RjPnXnEfLGJ6QXz3oIUtCk= 2023-09-06T18:54:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Pynp8NrbQabMkK3jYh9YCQ==', 'Content-Length': '3010540', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:DWb43RjPnXnEfLGJ6QXz3oIUtCk='} 2023-09-06T18:54:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.073 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.073 INFO:teuthology.orchestra.run.smithi099.stderr:YDCq1EcJ1shxd4KL33U9ZA== 2023-09-06T18:54:39.073 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.073 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.073 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/85 2023-09-06T18:54:39.074 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.074 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:H//lau2lDgmRT3ZT1hifxD4FDCA= 2023-09-06T18:54:39.074 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'YDCq1EcJ1shxd4KL33U9ZA==', 'Content-Length': '3011189', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:H//lau2lDgmRT3ZT1hifxD4FDCA='} 2023-09-06T18:54:39.074 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.074 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.075 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.075 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.075 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.075 INFO:teuthology.orchestra.run.smithi099.stderr:ApVRZRTGuJLDF4R7yT+TnA== 2023-09-06T18:54:39.075 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.075 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/93 2023-09-06T18:54:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:T0yfDwfwBTB5XlDGzeKSZspocV0= 2023-09-06T18:54:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ApVRZRTGuJLDF4R7yT+TnA==', 'Content-Length': '3008841', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:T0yfDwfwBTB5XlDGzeKSZspocV0='} 2023-09-06T18:54:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:FowqMMckz0VWUhuaNxe2nQ== 2023-09-06T18:54:39.078 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.078 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.078 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/60 2023-09-06T18:54:39.078 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.078 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1HpIPoejfKGUqWjAYirHX5A9szI= 2023-09-06T18:54:39.078 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'FowqMMckz0VWUhuaNxe2nQ==', 'Content-Length': '3011279', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1HpIPoejfKGUqWjAYirHX5A9szI='} 2023-09-06T18:54:39.079 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.079 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.079 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.079 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.079 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.080 INFO:teuthology.orchestra.run.smithi099.stderr:onB9dpXuID5Leh9ic6perg== 2023-09-06T18:54:39.080 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.080 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.080 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/71 2023-09-06T18:54:39.080 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.081 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Nu4fqXnsX73xXTrJxiVFnc1DqK8= 2023-09-06T18:54:39.081 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'onB9dpXuID5Leh9ic6perg==', 'Content-Length': '3010257', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Nu4fqXnsX73xXTrJxiVFnc1DqK8='} 2023-09-06T18:54:39.081 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.081 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.081 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:dy8UZfYnVbeBuu0eEEEAdw== 2023-09-06T18:54:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.083 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/59 2023-09-06T18:54:39.083 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.083 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:FMb0hLbWXab8IMiyuTx2Sd1t5k4= 2023-09-06T18:54:39.083 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'dy8UZfYnVbeBuu0eEEEAdw==', 'Content-Length': '3013085', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:FMb0hLbWXab8IMiyuTx2Sd1t5k4='} 2023-09-06T18:54:39.083 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.083 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.084 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.084 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.084 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.084 INFO:teuthology.orchestra.run.smithi099.stderr:O5q5DUvpDjeTWx2UQS0b/w== 2023-09-06T18:54:39.084 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.085 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.085 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/76 2023-09-06T18:54:39.085 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.085 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:klKXT0O66VTwU/oiBA1v0jgqpVU= 2023-09-06T18:54:39.085 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'O5q5DUvpDjeTWx2UQS0b/w==', 'Content-Length': '3012500', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:klKXT0O66VTwU/oiBA1v0jgqpVU='} 2023-09-06T18:54:39.086 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.086 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.086 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.086 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.086 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.086 INFO:teuthology.orchestra.run.smithi099.stderr:JY4+ecb2EmwJv//Rf3oChw== 2023-09-06T18:54:39.087 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.087 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.087 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/65 2023-09-06T18:54:39.087 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.087 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:e7Brq42o6Lust4KBxlCx+CThsJ0= 2023-09-06T18:54:39.088 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'JY4+ecb2EmwJv//Rf3oChw==', 'Content-Length': '3010799', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:e7Brq42o6Lust4KBxlCx+CThsJ0='} 2023-09-06T18:54:39.088 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.088 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.088 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.088 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:GJkR3zTdK3YwvIGvfcQsJA== 2023-09-06T18:54:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/94 2023-09-06T18:54:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.090 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1TCHMs8abxmMC5byV+PH4dfiIn8= 2023-09-06T18:54:39.090 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GJkR3zTdK3YwvIGvfcQsJA==', 'Content-Length': '3011083', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1TCHMs8abxmMC5byV+PH4dfiIn8='} 2023-09-06T18:54:39.090 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.090 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.090 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.091 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.091 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.091 INFO:teuthology.orchestra.run.smithi099.stderr:T5i766ZVJJ3eXXOUsc6lFA== 2023-09-06T18:54:39.091 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.092 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.092 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/63 2023-09-06T18:54:39.092 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.092 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:RYEZ13QtbXJqx9VanMv5lwSwvYo= 2023-09-06T18:54:39.092 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'T5i766ZVJJ3eXXOUsc6lFA==', 'Content-Length': '3010173', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:RYEZ13QtbXJqx9VanMv5lwSwvYo='} 2023-09-06T18:54:39.092 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.093 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.093 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.093 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.093 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.093 INFO:teuthology.orchestra.run.smithi099.stderr:ApVRZRTGuJLDF4R7yT+TnA== 2023-09-06T18:54:39.093 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.094 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.094 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/93 2023-09-06T18:54:39.094 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.094 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:T0yfDwfwBTB5XlDGzeKSZspocV0= 2023-09-06T18:54:39.094 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ApVRZRTGuJLDF4R7yT+TnA==', 'Content-Length': '3008841', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:T0yfDwfwBTB5XlDGzeKSZspocV0='} 2023-09-06T18:54:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:bzcWCe4Zs+Wz9mmNWMtfMA== 2023-09-06T18:54:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/86 2023-09-06T18:54:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:jytVWq34l+SSBNBapWZMC66VzHY= 2023-09-06T18:54:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'bzcWCe4Zs+Wz9mmNWMtfMA==', 'Content-Length': '3011876', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:jytVWq34l+SSBNBapWZMC66VzHY='} 2023-09-06T18:54:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.098 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.098 INFO:teuthology.orchestra.run.smithi099.stderr:3FruUTzNJSEDHJpOgQE21Q== 2023-09-06T18:54:39.098 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.098 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/98 2023-09-06T18:54:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:mNSz2fBFJ+HQyMBbfHNdv778AeE= 2023-09-06T18:54:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FruUTzNJSEDHJpOgQE21Q==', 'Content-Length': '3011202', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:mNSz2fBFJ+HQyMBbfHNdv778AeE='} 2023-09-06T18:54:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.100 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.100 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.100 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.100 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.100 INFO:teuthology.orchestra.run.smithi099.stderr:YDCq1EcJ1shxd4KL33U9ZA== 2023-09-06T18:54:39.100 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.101 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.101 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/85 2023-09-06T18:54:39.101 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.101 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:H//lau2lDgmRT3ZT1hifxD4FDCA= 2023-09-06T18:54:39.101 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'YDCq1EcJ1shxd4KL33U9ZA==', 'Content-Length': '3011189', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:H//lau2lDgmRT3ZT1hifxD4FDCA='} 2023-09-06T18:54:39.102 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.102 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.102 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.102 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.102 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.103 INFO:teuthology.orchestra.run.smithi099.stderr:az8m3gLwA3rfuOuGoIUlIg== 2023-09-06T18:54:39.103 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.103 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.103 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/79 2023-09-06T18:54:39.103 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.104 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:g3Ny3rsaOzsW2WwvyD68FiQb99s= 2023-09-06T18:54:39.104 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'az8m3gLwA3rfuOuGoIUlIg==', 'Content-Length': '3009702', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:g3Ny3rsaOzsW2WwvyD68FiQb99s='} 2023-09-06T18:54:39.104 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.104 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.104 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.104 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.105 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.105 INFO:teuthology.orchestra.run.smithi099.stderr:L+CnbtRBIsmcKjWa1bgqAQ== 2023-09-06T18:54:39.105 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.105 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.105 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/62 2023-09-06T18:54:39.106 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.106 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:YvmJ7d/T+PZLDfTssCi26WRDB6U= 2023-09-06T18:54:39.106 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'L+CnbtRBIsmcKjWa1bgqAQ==', 'Content-Length': '3012829', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:YvmJ7d/T+PZLDfTssCi26WRDB6U='} 2023-09-06T18:54:39.106 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.106 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.106 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.107 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.107 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.107 INFO:teuthology.orchestra.run.smithi099.stderr:TpxZegmN2A+sJ3sHtNR34g== 2023-09-06T18:54:39.107 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.107 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/81 2023-09-06T18:54:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6EPFOpoyygYmIMj+Rug1f0pP8YM= 2023-09-06T18:54:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TpxZegmN2A+sJ3sHtNR34g==', 'Content-Length': '3011797', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6EPFOpoyygYmIMj+Rug1f0pP8YM='} 2023-09-06T18:54:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.109 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.109 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.109 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.109 INFO:teuthology.orchestra.run.smithi099.stderr:SCQrX/nmwhId+jfW+hbFKA== 2023-09-06T18:54:39.109 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.109 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/82 2023-09-06T18:54:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:0g54EVFLjN5wdXtHGIks2sEB4/o= 2023-09-06T18:54:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'SCQrX/nmwhId+jfW+hbFKA==', 'Content-Length': '3013892', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:0g54EVFLjN5wdXtHGIks2sEB4/o='} 2023-09-06T18:54:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.111 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.111 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.111 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.111 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.112 INFO:teuthology.orchestra.run.smithi099.stderr:sTGY538hYoLTXfpqTlI66Q== 2023-09-06T18:54:39.112 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.112 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.112 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/64 2023-09-06T18:54:39.112 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.112 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:kPQ7Nw0ABJAdPoKDwfRqYz6cZE0= 2023-09-06T18:54:39.113 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'sTGY538hYoLTXfpqTlI66Q==', 'Content-Length': '3012286', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:kPQ7Nw0ABJAdPoKDwfRqYz6cZE0='} 2023-09-06T18:54:39.113 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.113 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.113 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.113 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:vUxwqUKsZaB1t8UY3dtHOg== 2023-09-06T18:54:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/87 2023-09-06T18:54:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:EBRZ6AH9z0Bo+QZREF5CA5CZ9XA= 2023-09-06T18:54:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'vUxwqUKsZaB1t8UY3dtHOg==', 'Content-Length': '3009715', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:EBRZ6AH9z0Bo+QZREF5CA5CZ9XA='} 2023-09-06T18:54:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:58/syNvnMEysIcHSAd8NBg== 2023-09-06T18:54:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.117 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.117 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/72 2023-09-06T18:54:39.117 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.117 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:34itloabOLBXXfokpjOSbKGYcPY= 2023-09-06T18:54:39.117 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '58/syNvnMEysIcHSAd8NBg==', 'Content-Length': '3012528', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:34itloabOLBXXfokpjOSbKGYcPY='} 2023-09-06T18:54:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:13KPoJjsWZmb48Fg0K6R/g== 2023-09-06T18:54:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/75 2023-09-06T18:54:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:8wq87cdlwIhv6AMzvkexpnLie1o= 2023-09-06T18:54:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '13KPoJjsWZmb48Fg0K6R/g==', 'Content-Length': '3010936', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:8wq87cdlwIhv6AMzvkexpnLie1o='} 2023-09-06T18:54:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.121 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.121 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.121 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.121 INFO:teuthology.orchestra.run.smithi099.stderr:az8m3gLwA3rfuOuGoIUlIg== 2023-09-06T18:54:39.121 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.121 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.122 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/79 2023-09-06T18:54:39.122 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.122 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:g3Ny3rsaOzsW2WwvyD68FiQb99s= 2023-09-06T18:54:39.122 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'az8m3gLwA3rfuOuGoIUlIg==', 'Content-Length': '3009702', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:g3Ny3rsaOzsW2WwvyD68FiQb99s='} 2023-09-06T18:54:39.122 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.123 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.123 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.123 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.123 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.123 INFO:teuthology.orchestra.run.smithi099.stderr:d1WW15syDrKs3UdrMlkfWg== 2023-09-06T18:54:39.124 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.124 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.124 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/92 2023-09-06T18:54:39.124 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.124 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:4bsf40rLqAbfE0v5i1qQUo03ox4= 2023-09-06T18:54:39.124 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'd1WW15syDrKs3UdrMlkfWg==', 'Content-Length': '3010384', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:4bsf40rLqAbfE0v5i1qQUo03ox4='} 2023-09-06T18:54:39.125 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.125 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.125 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.125 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.126 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.126 INFO:teuthology.orchestra.run.smithi099.stderr:1Wi/eLV38vm0BZtxAI2RNA== 2023-09-06T18:54:39.126 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.126 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.126 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/97 2023-09-06T18:54:39.126 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.127 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:dNHETCwYqsClR6tlXTHGA3vYsm8= 2023-09-06T18:54:39.127 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1Wi/eLV38vm0BZtxAI2RNA==', 'Content-Length': '3011110', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:dNHETCwYqsClR6tlXTHGA3vYsm8='} 2023-09-06T18:54:39.127 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.128 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.128 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.128 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.128 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.128 INFO:teuthology.orchestra.run.smithi099.stderr:C/iQp/RnJcz00qfhYKVejg== 2023-09-06T18:54:39.128 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.129 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.129 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/83 2023-09-06T18:54:39.129 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.129 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VBQiPvpgkgY0eGav2GjAQBMnjks= 2023-09-06T18:54:39.129 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C/iQp/RnJcz00qfhYKVejg==', 'Content-Length': '3008624', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VBQiPvpgkgY0eGav2GjAQBMnjks='} 2023-09-06T18:54:39.129 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.130 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.130 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.130 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.130 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.130 INFO:teuthology.orchestra.run.smithi099.stderr:dy8UZfYnVbeBuu0eEEEAdw== 2023-09-06T18:54:39.131 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.131 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.131 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/59 2023-09-06T18:54:39.131 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.131 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:FMb0hLbWXab8IMiyuTx2Sd1t5k4= 2023-09-06T18:54:39.131 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'dy8UZfYnVbeBuu0eEEEAdw==', 'Content-Length': '3013085', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:FMb0hLbWXab8IMiyuTx2Sd1t5k4='} 2023-09-06T18:54:39.132 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.132 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.132 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.132 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.132 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.133 INFO:teuthology.orchestra.run.smithi099.stderr:6Mq+4Y9CKsucB1afANko4g== 2023-09-06T18:54:39.133 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.133 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.133 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/90 2023-09-06T18:54:39.133 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.133 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1hcMZJN2zAi4C/ciWwcDP+nO928= 2023-09-06T18:54:39.134 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '6Mq+4Y9CKsucB1afANko4g==', 'Content-Length': '3013824', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1hcMZJN2zAi4C/ciWwcDP+nO928='} 2023-09-06T18:54:39.134 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.134 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.134 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.135 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.135 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.135 INFO:teuthology.orchestra.run.smithi099.stderr:C6D+oxut6krwvUPn9E6aqQ== 2023-09-06T18:54:39.135 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.135 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/68 2023-09-06T18:54:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:CtTT22ewDAnC9YahPSmJI1/2Xt0= 2023-09-06T18:54:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C6D+oxut6krwvUPn9E6aqQ==', 'Content-Length': '3009821', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:CtTT22ewDAnC9YahPSmJI1/2Xt0='} 2023-09-06T18:54:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.137 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.137 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.137 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.137 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.137 INFO:teuthology.orchestra.run.smithi099.stderr:CExctBxSvJfXGKs/K/T9MQ== 2023-09-06T18:54:39.137 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/67 2023-09-06T18:54:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Kg+8r2tVv0ClHQIwVtGnpmy4f0A= 2023-09-06T18:54:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'CExctBxSvJfXGKs/K/T9MQ==', 'Content-Length': '3011407', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Kg+8r2tVv0ClHQIwVtGnpmy4f0A='} 2023-09-06T18:54:39.139 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.139 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.139 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.139 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.139 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.139 INFO:teuthology.orchestra.run.smithi099.stderr:FowqMMckz0VWUhuaNxe2nQ== 2023-09-06T18:54:39.140 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.140 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.140 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/60 2023-09-06T18:54:39.140 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.140 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1HpIPoejfKGUqWjAYirHX5A9szI= 2023-09-06T18:54:39.141 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'FowqMMckz0VWUhuaNxe2nQ==', 'Content-Length': '3011279', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1HpIPoejfKGUqWjAYirHX5A9szI='} 2023-09-06T18:54:39.141 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.141 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.141 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.141 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.142 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.142 INFO:teuthology.orchestra.run.smithi099.stderr:O5q5DUvpDjeTWx2UQS0b/w== 2023-09-06T18:54:39.142 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.142 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.142 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/76 2023-09-06T18:54:39.142 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.143 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:klKXT0O66VTwU/oiBA1v0jgqpVU= 2023-09-06T18:54:39.143 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'O5q5DUvpDjeTWx2UQS0b/w==', 'Content-Length': '3012500', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:klKXT0O66VTwU/oiBA1v0jgqpVU='} 2023-09-06T18:54:39.143 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.143 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.143 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.143 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.144 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.144 INFO:teuthology.orchestra.run.smithi099.stderr:Pynp8NrbQabMkK3jYh9YCQ== 2023-09-06T18:54:39.144 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.144 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.144 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/84 2023-09-06T18:54:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:DWb43RjPnXnEfLGJ6QXz3oIUtCk= 2023-09-06T18:54:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Pynp8NrbQabMkK3jYh9YCQ==', 'Content-Length': '3010540', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:DWb43RjPnXnEfLGJ6QXz3oIUtCk='} 2023-09-06T18:54:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.146 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.146 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.146 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.146 INFO:teuthology.orchestra.run.smithi099.stderr:YDCq1EcJ1shxd4KL33U9ZA== 2023-09-06T18:54:39.146 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.147 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.147 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/85 2023-09-06T18:54:39.147 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.147 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:H//lau2lDgmRT3ZT1hifxD4FDCA= 2023-09-06T18:54:39.147 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'YDCq1EcJ1shxd4KL33U9ZA==', 'Content-Length': '3011189', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:H//lau2lDgmRT3ZT1hifxD4FDCA='} 2023-09-06T18:54:39.148 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.148 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.148 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.148 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.148 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.148 INFO:teuthology.orchestra.run.smithi099.stderr:8B3cc/t8TpRZF5cCer6dEQ== 2023-09-06T18:54:39.149 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.149 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.149 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/77 2023-09-06T18:54:39.149 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.149 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VWVCrMVG65GwMoPOxodu30Gx5IY= 2023-09-06T18:54:39.150 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '8B3cc/t8TpRZF5cCer6dEQ==', 'Content-Length': '3008801', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VWVCrMVG65GwMoPOxodu30Gx5IY='} 2023-09-06T18:54:39.150 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.150 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.150 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.150 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.151 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.151 INFO:teuthology.orchestra.run.smithi099.stderr:MILNOxKsGQkBzSCgDskNTA== 2023-09-06T18:54:39.151 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.151 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.151 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/80 2023-09-06T18:54:39.152 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.152 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:5sx0j0PgApPbvZFYmcsddlB3Dig= 2023-09-06T18:54:39.152 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MILNOxKsGQkBzSCgDskNTA==', 'Content-Length': '3009370', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:5sx0j0PgApPbvZFYmcsddlB3Dig='} 2023-09-06T18:54:39.152 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.152 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.152 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.153 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.153 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.153 INFO:teuthology.orchestra.run.smithi099.stderr:wH4O4G0cUa3RVZTZL8Bx8g== 2023-09-06T18:54:39.153 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.153 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.154 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/74 2023-09-06T18:54:39.154 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.154 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:oXpzU2EWQ1vcOuvfW97TE3ZNzJg= 2023-09-06T18:54:39.154 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'wH4O4G0cUa3RVZTZL8Bx8g==', 'Content-Length': '3009021', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:oXpzU2EWQ1vcOuvfW97TE3ZNzJg='} 2023-09-06T18:54:39.154 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.155 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.155 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.155 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.155 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.155 INFO:teuthology.orchestra.run.smithi099.stderr:T5i766ZVJJ3eXXOUsc6lFA== 2023-09-06T18:54:39.156 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.156 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.156 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/63 2023-09-06T18:54:39.156 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.156 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:RYEZ13QtbXJqx9VanMv5lwSwvYo= 2023-09-06T18:54:39.157 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'T5i766ZVJJ3eXXOUsc6lFA==', 'Content-Length': '3010173', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:RYEZ13QtbXJqx9VanMv5lwSwvYo='} 2023-09-06T18:54:39.157 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.157 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.157 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.157 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.158 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.158 INFO:teuthology.orchestra.run.smithi099.stderr:ZTBtXbDjEjBhg8rpL9GNdw== 2023-09-06T18:54:39.158 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.158 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.158 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/91 2023-09-06T18:54:39.158 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.159 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:nQP1DEadb8eksNgyNVzpg8Nd0as= 2023-09-06T18:54:39.159 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ZTBtXbDjEjBhg8rpL9GNdw==', 'Content-Length': '3011579', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:nQP1DEadb8eksNgyNVzpg8Nd0as='} 2023-09-06T18:54:39.159 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.159 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.160 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.160 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.160 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.160 INFO:teuthology.orchestra.run.smithi099.stderr:onB9dpXuID5Leh9ic6perg== 2023-09-06T18:54:39.160 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.161 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.161 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/71 2023-09-06T18:54:39.161 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.161 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Nu4fqXnsX73xXTrJxiVFnc1DqK8= 2023-09-06T18:54:39.161 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'onB9dpXuID5Leh9ic6perg==', 'Content-Length': '3010257', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Nu4fqXnsX73xXTrJxiVFnc1DqK8='} 2023-09-06T18:54:39.162 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.162 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.162 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.162 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.162 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.163 INFO:teuthology.orchestra.run.smithi099.stderr:3FruUTzNJSEDHJpOgQE21Q== 2023-09-06T18:54:39.163 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.163 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.163 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/98 2023-09-06T18:54:39.164 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.164 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:mNSz2fBFJ+HQyMBbfHNdv778AeE= 2023-09-06T18:54:39.164 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FruUTzNJSEDHJpOgQE21Q==', 'Content-Length': '3011202', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:mNSz2fBFJ+HQyMBbfHNdv778AeE='} 2023-09-06T18:54:39.164 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.164 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.165 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.165 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.165 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.165 INFO:teuthology.orchestra.run.smithi099.stderr:vUxwqUKsZaB1t8UY3dtHOg== 2023-09-06T18:54:39.165 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.166 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.166 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/87 2023-09-06T18:54:39.166 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.166 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:EBRZ6AH9z0Bo+QZREF5CA5CZ9XA= 2023-09-06T18:54:39.166 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'vUxwqUKsZaB1t8UY3dtHOg==', 'Content-Length': '3009715', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:EBRZ6AH9z0Bo+QZREF5CA5CZ9XA='} 2023-09-06T18:54:39.167 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.167 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.167 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.167 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.167 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.168 INFO:teuthology.orchestra.run.smithi099.stderr:H/7PVRaBhaiTgBGwsocA9g== 2023-09-06T18:54:39.168 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.168 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.168 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/66 2023-09-06T18:54:39.168 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.168 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:36FUK1BTY8M1HxewVhurZeS63eU= 2023-09-06T18:54:39.169 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'H/7PVRaBhaiTgBGwsocA9g==', 'Content-Length': '3010803', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:36FUK1BTY8M1HxewVhurZeS63eU='} 2023-09-06T18:54:39.169 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.169 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.169 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.169 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.170 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.170 INFO:teuthology.orchestra.run.smithi099.stderr:LInDI/4s7vtCmmUdl2GZgQ== 2023-09-06T18:54:39.170 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.170 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.170 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/78 2023-09-06T18:54:39.171 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.171 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:wTYx6jGw8NbNYPV0mbzgPssE+PE= 2023-09-06T18:54:39.171 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'LInDI/4s7vtCmmUdl2GZgQ==', 'Content-Length': '3012430', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:wTYx6jGw8NbNYPV0mbzgPssE+PE='} 2023-09-06T18:54:39.173 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.173 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.173 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.173 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.173 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.174 INFO:teuthology.orchestra.run.smithi099.stderr:d1WW15syDrKs3UdrMlkfWg== 2023-09-06T18:54:39.174 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.174 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.174 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/92 2023-09-06T18:54:39.174 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.175 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:4bsf40rLqAbfE0v5i1qQUo03ox4= 2023-09-06T18:54:39.175 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'd1WW15syDrKs3UdrMlkfWg==', 'Content-Length': '3010384', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:4bsf40rLqAbfE0v5i1qQUo03ox4='} 2023-09-06T18:54:39.175 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.175 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.175 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.176 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.176 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.176 INFO:teuthology.orchestra.run.smithi099.stderr:TloLB9lPBB2/TtlB2Gy1wA== 2023-09-06T18:54:39.176 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.176 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.177 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/69 2023-09-06T18:54:39.177 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.177 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6qcEOM3G1/zk11ieYYG76Z/6oiA= 2023-09-06T18:54:39.177 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TloLB9lPBB2/TtlB2Gy1wA==', 'Content-Length': '3009018', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6qcEOM3G1/zk11ieYYG76Z/6oiA='} 2023-09-06T18:54:39.177 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.178 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.178 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.178 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.178 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.178 INFO:teuthology.orchestra.run.smithi099.stderr:OkRpNuHoekxqIV5d/AEmQw== 2023-09-06T18:54:39.179 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.179 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.179 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/96 2023-09-06T18:54:39.179 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.179 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VNYADlT08pfUnJaj4kH1qEp0w0M= 2023-09-06T18:54:39.180 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'OkRpNuHoekxqIV5d/AEmQw==', 'Content-Length': '3010153', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VNYADlT08pfUnJaj4kH1qEp0w0M='} 2023-09-06T18:54:39.180 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.180 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.180 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.180 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.181 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.181 INFO:teuthology.orchestra.run.smithi099.stderr:ACjFwH5NGlKEdPWFDs0Uug== 2023-09-06T18:54:39.181 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.181 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.181 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/70 2023-09-06T18:54:39.181 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.182 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Gvl3cUJ2iRK2IcsrZZrixV76LeQ= 2023-09-06T18:54:39.182 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ACjFwH5NGlKEdPWFDs0Uug==', 'Content-Length': '3011260', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Gvl3cUJ2iRK2IcsrZZrixV76LeQ='} 2023-09-06T18:54:39.182 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.182 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.182 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.183 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.183 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.183 INFO:teuthology.orchestra.run.smithi099.stderr:H/7PVRaBhaiTgBGwsocA9g== 2023-09-06T18:54:39.183 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.183 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.183 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/66 2023-09-06T18:54:39.184 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.184 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:36FUK1BTY8M1HxewVhurZeS63eU= 2023-09-06T18:54:39.184 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'H/7PVRaBhaiTgBGwsocA9g==', 'Content-Length': '3010803', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:36FUK1BTY8M1HxewVhurZeS63eU='} 2023-09-06T18:54:39.184 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.184 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.185 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.185 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.185 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.185 INFO:teuthology.orchestra.run.smithi099.stderr:SCQrX/nmwhId+jfW+hbFKA== 2023-09-06T18:54:39.185 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.186 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.186 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/82 2023-09-06T18:54:39.186 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.186 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:0g54EVFLjN5wdXtHGIks2sEB4/o= 2023-09-06T18:54:39.186 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'SCQrX/nmwhId+jfW+hbFKA==', 'Content-Length': '3013892', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:0g54EVFLjN5wdXtHGIks2sEB4/o='} 2023-09-06T18:54:39.187 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.187 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.187 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.187 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.187 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.188 INFO:teuthology.orchestra.run.smithi099.stderr:4lhMsR4INMw5LUlW+fjijg== 2023-09-06T18:54:39.188 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.188 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.188 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/89 2023-09-06T18:54:39.188 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.189 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:vnxVSmpqWU4SKgYVVv2CZeWLgTU= 2023-09-06T18:54:39.189 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '4lhMsR4INMw5LUlW+fjijg==', 'Content-Length': '3008833', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:vnxVSmpqWU4SKgYVVv2CZeWLgTU='} 2023-09-06T18:54:39.189 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.189 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.189 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.190 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.190 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.190 INFO:teuthology.orchestra.run.smithi099.stderr:bzcWCe4Zs+Wz9mmNWMtfMA== 2023-09-06T18:54:39.190 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.190 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.191 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/86 2023-09-06T18:54:39.191 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.191 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:jytVWq34l+SSBNBapWZMC66VzHY= 2023-09-06T18:54:39.191 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'bzcWCe4Zs+Wz9mmNWMtfMA==', 'Content-Length': '3011876', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:jytVWq34l+SSBNBapWZMC66VzHY='} 2023-09-06T18:54:39.191 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.191 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.192 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.192 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.192 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.192 INFO:teuthology.orchestra.run.smithi099.stderr:GJkR3zTdK3YwvIGvfcQsJA== 2023-09-06T18:54:39.192 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.192 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.193 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/94 2023-09-06T18:54:39.193 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.193 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1TCHMs8abxmMC5byV+PH4dfiIn8= 2023-09-06T18:54:39.193 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GJkR3zTdK3YwvIGvfcQsJA==', 'Content-Length': '3011083', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1TCHMs8abxmMC5byV+PH4dfiIn8='} 2023-09-06T18:54:39.193 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.194 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.194 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.194 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.194 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.194 INFO:teuthology.orchestra.run.smithi099.stderr:sTGY538hYoLTXfpqTlI66Q== 2023-09-06T18:54:39.195 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.195 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.195 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/64 2023-09-06T18:54:39.195 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.195 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:kPQ7Nw0ABJAdPoKDwfRqYz6cZE0= 2023-09-06T18:54:39.196 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'sTGY538hYoLTXfpqTlI66Q==', 'Content-Length': '3012286', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:kPQ7Nw0ABJAdPoKDwfRqYz6cZE0='} 2023-09-06T18:54:39.196 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.196 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.196 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.196 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.197 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.197 INFO:teuthology.orchestra.run.smithi099.stderr:1Wi/eLV38vm0BZtxAI2RNA== 2023-09-06T18:54:39.197 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.197 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.197 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/97 2023-09-06T18:54:39.197 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.198 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:dNHETCwYqsClR6tlXTHGA3vYsm8= 2023-09-06T18:54:39.198 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1Wi/eLV38vm0BZtxAI2RNA==', 'Content-Length': '3011110', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:dNHETCwYqsClR6tlXTHGA3vYsm8='} 2023-09-06T18:54:39.198 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.198 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.198 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.199 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.199 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.199 INFO:teuthology.orchestra.run.smithi099.stderr:TloLB9lPBB2/TtlB2Gy1wA== 2023-09-06T18:54:39.199 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.199 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.200 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/69 2023-09-06T18:54:39.200 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.200 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6qcEOM3G1/zk11ieYYG76Z/6oiA= 2023-09-06T18:54:39.200 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TloLB9lPBB2/TtlB2Gy1wA==', 'Content-Length': '3009018', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6qcEOM3G1/zk11ieYYG76Z/6oiA='} 2023-09-06T18:54:39.200 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.201 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.201 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.201 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.201 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.201 INFO:teuthology.orchestra.run.smithi099.stderr:IlteDkzUvp19xkYkWXYCEw== 2023-09-06T18:54:39.202 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.202 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.202 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/95 2023-09-06T18:54:39.202 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.202 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:qU8kqZa6Kd3gi2mTN1yByHiSPhs= 2023-09-06T18:54:39.203 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'IlteDkzUvp19xkYkWXYCEw==', 'Content-Length': '3010755', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:qU8kqZa6Kd3gi2mTN1yByHiSPhs='} 2023-09-06T18:54:39.203 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.203 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.203 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.203 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.204 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.204 INFO:teuthology.orchestra.run.smithi099.stderr:58/syNvnMEysIcHSAd8NBg== 2023-09-06T18:54:39.204 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.204 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.204 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/72 2023-09-06T18:54:39.205 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.205 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:34itloabOLBXXfokpjOSbKGYcPY= 2023-09-06T18:54:39.205 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '58/syNvnMEysIcHSAd8NBg==', 'Content-Length': '3012528', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:34itloabOLBXXfokpjOSbKGYcPY='} 2023-09-06T18:54:39.205 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.205 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.206 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.206 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.206 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.206 INFO:teuthology.orchestra.run.smithi099.stderr:JY4+ecb2EmwJv//Rf3oChw== 2023-09-06T18:54:39.206 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.207 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.207 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/65 2023-09-06T18:54:39.207 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.207 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:e7Brq42o6Lust4KBxlCx+CThsJ0= 2023-09-06T18:54:39.207 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'JY4+ecb2EmwJv//Rf3oChw==', 'Content-Length': '3010799', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:e7Brq42o6Lust4KBxlCx+CThsJ0='} 2023-09-06T18:54:39.207 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.208 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.208 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.208 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.208 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.208 INFO:teuthology.orchestra.run.smithi099.stderr:ivwZ7Vdh/5AezzspgtVxkA== 2023-09-06T18:54:39.209 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.209 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.209 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/73 2023-09-06T18:54:39.209 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.209 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:rAgrUjCY/2UU1peWmmZ9tsZx824= 2023-09-06T18:54:39.210 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ivwZ7Vdh/5AezzspgtVxkA==', 'Content-Length': '3008097', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:rAgrUjCY/2UU1peWmmZ9tsZx824='} 2023-09-06T18:54:39.210 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.210 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.210 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.210 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.211 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.211 INFO:teuthology.orchestra.run.smithi099.stderr:sTGY538hYoLTXfpqTlI66Q== 2023-09-06T18:54:39.211 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.211 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.211 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/64 2023-09-06T18:54:39.212 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.212 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:kPQ7Nw0ABJAdPoKDwfRqYz6cZE0= 2023-09-06T18:54:39.212 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'sTGY538hYoLTXfpqTlI66Q==', 'Content-Length': '3012286', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:kPQ7Nw0ABJAdPoKDwfRqYz6cZE0='} 2023-09-06T18:54:39.212 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.212 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.213 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.213 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.213 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.213 INFO:teuthology.orchestra.run.smithi099.stderr:Pynp8NrbQabMkK3jYh9YCQ== 2023-09-06T18:54:39.213 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.214 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.214 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/84 2023-09-06T18:54:39.214 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.214 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:DWb43RjPnXnEfLGJ6QXz3oIUtCk= 2023-09-06T18:54:39.214 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Pynp8NrbQabMkK3jYh9YCQ==', 'Content-Length': '3010540', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:DWb43RjPnXnEfLGJ6QXz3oIUtCk='} 2023-09-06T18:54:39.215 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.215 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.215 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.215 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.215 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.216 INFO:teuthology.orchestra.run.smithi099.stderr:13KPoJjsWZmb48Fg0K6R/g== 2023-09-06T18:54:39.216 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.216 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.216 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/75 2023-09-06T18:54:39.216 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.217 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:8wq87cdlwIhv6AMzvkexpnLie1o= 2023-09-06T18:54:39.217 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '13KPoJjsWZmb48Fg0K6R/g==', 'Content-Length': '3010936', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:8wq87cdlwIhv6AMzvkexpnLie1o='} 2023-09-06T18:54:39.217 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.217 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.217 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.218 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.218 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.218 INFO:teuthology.orchestra.run.smithi099.stderr:YDCq1EcJ1shxd4KL33U9ZA== 2023-09-06T18:54:39.218 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.218 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.219 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/85 2023-09-06T18:54:39.219 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.219 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:H//lau2lDgmRT3ZT1hifxD4FDCA= 2023-09-06T18:54:39.219 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'YDCq1EcJ1shxd4KL33U9ZA==', 'Content-Length': '3011189', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:H//lau2lDgmRT3ZT1hifxD4FDCA='} 2023-09-06T18:54:39.219 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.220 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.220 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.220 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.220 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.220 INFO:teuthology.orchestra.run.smithi099.stderr:MwPOSAa17PNqNyNds46rPQ== 2023-09-06T18:54:39.221 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.221 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.221 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/88 2023-09-06T18:54:39.221 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.221 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:PYSmmu8GGwgwE32n73PXNZEqkko= 2023-09-06T18:54:39.222 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MwPOSAa17PNqNyNds46rPQ==', 'Content-Length': '3010515', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:PYSmmu8GGwgwE32n73PXNZEqkko='} 2023-09-06T18:54:39.222 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.222 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.222 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.222 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.222 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.223 INFO:teuthology.orchestra.run.smithi099.stderr:GJkR3zTdK3YwvIGvfcQsJA== 2023-09-06T18:54:39.223 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.223 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.223 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/94 2023-09-06T18:54:39.223 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.224 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1TCHMs8abxmMC5byV+PH4dfiIn8= 2023-09-06T18:54:39.224 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GJkR3zTdK3YwvIGvfcQsJA==', 'Content-Length': '3011083', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1TCHMs8abxmMC5byV+PH4dfiIn8='} 2023-09-06T18:54:39.224 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.224 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.224 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.225 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.225 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.225 INFO:teuthology.orchestra.run.smithi099.stderr:hJWNhlOi09qLpvsl0uu7Cg== 2023-09-06T18:54:39.225 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.225 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.225 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/99 2023-09-06T18:54:39.226 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.226 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Atvf3JbMyzqzFA+QlmjLbhjVYnk= 2023-09-06T18:54:39.226 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hJWNhlOi09qLpvsl0uu7Cg==', 'Content-Length': '3010645', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Atvf3JbMyzqzFA+QlmjLbhjVYnk='} 2023-09-06T18:54:39.226 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.227 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.227 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.227 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.227 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.227 INFO:teuthology.orchestra.run.smithi099.stderr:OkRpNuHoekxqIV5d/AEmQw== 2023-09-06T18:54:39.227 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.228 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.228 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/96 2023-09-06T18:54:39.228 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.228 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VNYADlT08pfUnJaj4kH1qEp0w0M= 2023-09-06T18:54:39.228 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'OkRpNuHoekxqIV5d/AEmQw==', 'Content-Length': '3010153', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VNYADlT08pfUnJaj4kH1qEp0w0M='} 2023-09-06T18:54:39.229 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.229 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.229 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.229 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.229 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.230 INFO:teuthology.orchestra.run.smithi099.stderr:O5q5DUvpDjeTWx2UQS0b/w== 2023-09-06T18:54:39.230 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.230 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.230 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/76 2023-09-06T18:54:39.230 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.231 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:klKXT0O66VTwU/oiBA1v0jgqpVU= 2023-09-06T18:54:39.231 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'O5q5DUvpDjeTWx2UQS0b/w==', 'Content-Length': '3012500', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:klKXT0O66VTwU/oiBA1v0jgqpVU='} 2023-09-06T18:54:39.231 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.231 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.231 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.232 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.232 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.232 INFO:teuthology.orchestra.run.smithi099.stderr:ApVRZRTGuJLDF4R7yT+TnA== 2023-09-06T18:54:39.232 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.232 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.232 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/93 2023-09-06T18:54:39.233 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.233 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:T0yfDwfwBTB5XlDGzeKSZspocV0= 2023-09-06T18:54:39.233 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ApVRZRTGuJLDF4R7yT+TnA==', 'Content-Length': '3008841', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:T0yfDwfwBTB5XlDGzeKSZspocV0='} 2023-09-06T18:54:39.233 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.233 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.233 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.234 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.234 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.234 INFO:teuthology.orchestra.run.smithi099.stderr:C/iQp/RnJcz00qfhYKVejg== 2023-09-06T18:54:39.234 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.234 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.235 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/83 2023-09-06T18:54:39.235 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.235 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VBQiPvpgkgY0eGav2GjAQBMnjks= 2023-09-06T18:54:39.235 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C/iQp/RnJcz00qfhYKVejg==', 'Content-Length': '3008624', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VBQiPvpgkgY0eGav2GjAQBMnjks='} 2023-09-06T18:54:39.236 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.236 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.236 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.236 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.236 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.236 INFO:teuthology.orchestra.run.smithi099.stderr:d1WW15syDrKs3UdrMlkfWg== 2023-09-06T18:54:39.237 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.237 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.237 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/92 2023-09-06T18:54:39.237 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.237 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:4bsf40rLqAbfE0v5i1qQUo03ox4= 2023-09-06T18:54:39.237 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'd1WW15syDrKs3UdrMlkfWg==', 'Content-Length': '3010384', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:4bsf40rLqAbfE0v5i1qQUo03ox4='} 2023-09-06T18:54:39.238 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.238 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.238 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.238 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.238 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.239 INFO:teuthology.orchestra.run.smithi099.stderr:3FruUTzNJSEDHJpOgQE21Q== 2023-09-06T18:54:39.239 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.239 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.239 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/98 2023-09-06T18:54:39.239 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.240 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:mNSz2fBFJ+HQyMBbfHNdv778AeE= 2023-09-06T18:54:39.240 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FruUTzNJSEDHJpOgQE21Q==', 'Content-Length': '3011202', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:mNSz2fBFJ+HQyMBbfHNdv778AeE='} 2023-09-06T18:54:39.240 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.240 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.240 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.240 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.241 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.241 INFO:teuthology.orchestra.run.smithi099.stderr:MILNOxKsGQkBzSCgDskNTA== 2023-09-06T18:54:39.241 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.241 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.241 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/80 2023-09-06T18:54:39.242 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.242 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:5sx0j0PgApPbvZFYmcsddlB3Dig= 2023-09-06T18:54:39.242 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MILNOxKsGQkBzSCgDskNTA==', 'Content-Length': '3009370', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:5sx0j0PgApPbvZFYmcsddlB3Dig='} 2023-09-06T18:54:39.242 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.242 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.243 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.243 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.243 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.243 INFO:teuthology.orchestra.run.smithi099.stderr:O5q5DUvpDjeTWx2UQS0b/w== 2023-09-06T18:54:39.243 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.243 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.244 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/76 2023-09-06T18:54:39.244 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.244 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:klKXT0O66VTwU/oiBA1v0jgqpVU= 2023-09-06T18:54:39.244 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'O5q5DUvpDjeTWx2UQS0b/w==', 'Content-Length': '3012500', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:klKXT0O66VTwU/oiBA1v0jgqpVU='} 2023-09-06T18:54:39.244 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.244 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.245 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.245 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.245 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.245 INFO:teuthology.orchestra.run.smithi099.stderr:13KPoJjsWZmb48Fg0K6R/g== 2023-09-06T18:54:39.245 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.246 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.246 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/75 2023-09-06T18:54:39.246 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.246 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:8wq87cdlwIhv6AMzvkexpnLie1o= 2023-09-06T18:54:39.246 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '13KPoJjsWZmb48Fg0K6R/g==', 'Content-Length': '3010936', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:8wq87cdlwIhv6AMzvkexpnLie1o='} 2023-09-06T18:54:39.247 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.247 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.247 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.247 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.247 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.248 INFO:teuthology.orchestra.run.smithi099.stderr:JY4+ecb2EmwJv//Rf3oChw== 2023-09-06T18:54:39.248 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.248 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.248 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/65 2023-09-06T18:54:39.248 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.248 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:e7Brq42o6Lust4KBxlCx+CThsJ0= 2023-09-06T18:54:39.249 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'JY4+ecb2EmwJv//Rf3oChw==', 'Content-Length': '3010799', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:e7Brq42o6Lust4KBxlCx+CThsJ0='} 2023-09-06T18:54:39.249 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.249 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.249 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.249 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.249 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.250 INFO:teuthology.orchestra.run.smithi099.stderr:dy8UZfYnVbeBuu0eEEEAdw== 2023-09-06T18:54:39.250 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.250 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.250 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/59 2023-09-06T18:54:39.250 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.251 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:FMb0hLbWXab8IMiyuTx2Sd1t5k4= 2023-09-06T18:54:39.251 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'dy8UZfYnVbeBuu0eEEEAdw==', 'Content-Length': '3013085', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:FMb0hLbWXab8IMiyuTx2Sd1t5k4='} 2023-09-06T18:54:39.251 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.251 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.251 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.252 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.252 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.252 INFO:teuthology.orchestra.run.smithi099.stderr:CExctBxSvJfXGKs/K/T9MQ== 2023-09-06T18:54:39.252 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.252 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.253 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/67 2023-09-06T18:54:39.253 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.253 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Kg+8r2tVv0ClHQIwVtGnpmy4f0A= 2023-09-06T18:54:39.253 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'CExctBxSvJfXGKs/K/T9MQ==', 'Content-Length': '3011407', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Kg+8r2tVv0ClHQIwVtGnpmy4f0A='} 2023-09-06T18:54:39.253 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.254 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.254 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.254 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.254 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.254 INFO:teuthology.orchestra.run.smithi099.stderr:FowqMMckz0VWUhuaNxe2nQ== 2023-09-06T18:54:39.255 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.255 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.255 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/60 2023-09-06T18:54:39.255 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.255 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1HpIPoejfKGUqWjAYirHX5A9szI= 2023-09-06T18:54:39.255 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'FowqMMckz0VWUhuaNxe2nQ==', 'Content-Length': '3011279', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1HpIPoejfKGUqWjAYirHX5A9szI='} 2023-09-06T18:54:39.256 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.256 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.256 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.256 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.257 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.257 INFO:teuthology.orchestra.run.smithi099.stderr:CExctBxSvJfXGKs/K/T9MQ== 2023-09-06T18:54:39.257 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.257 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.257 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/67 2023-09-06T18:54:39.258 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.258 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Kg+8r2tVv0ClHQIwVtGnpmy4f0A= 2023-09-06T18:54:39.258 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'CExctBxSvJfXGKs/K/T9MQ==', 'Content-Length': '3011407', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Kg+8r2tVv0ClHQIwVtGnpmy4f0A='} 2023-09-06T18:54:39.258 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.258 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.259 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.259 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.259 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.259 INFO:teuthology.orchestra.run.smithi099.stderr:YDCq1EcJ1shxd4KL33U9ZA== 2023-09-06T18:54:39.259 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.259 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.260 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/85 2023-09-06T18:54:39.260 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.260 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:H//lau2lDgmRT3ZT1hifxD4FDCA= 2023-09-06T18:54:39.260 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'YDCq1EcJ1shxd4KL33U9ZA==', 'Content-Length': '3011189', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:H//lau2lDgmRT3ZT1hifxD4FDCA='} 2023-09-06T18:54:39.260 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.261 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.261 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.261 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.261 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.261 INFO:teuthology.orchestra.run.smithi099.stderr:L+CnbtRBIsmcKjWa1bgqAQ== 2023-09-06T18:54:39.262 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.262 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.262 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/62 2023-09-06T18:54:39.262 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.262 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:YvmJ7d/T+PZLDfTssCi26WRDB6U= 2023-09-06T18:54:39.263 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'L+CnbtRBIsmcKjWa1bgqAQ==', 'Content-Length': '3012829', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:YvmJ7d/T+PZLDfTssCi26WRDB6U='} 2023-09-06T18:54:39.263 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.263 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.263 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.263 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.263 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.264 INFO:teuthology.orchestra.run.smithi099.stderr:T5i766ZVJJ3eXXOUsc6lFA== 2023-09-06T18:54:39.264 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.264 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.264 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/63 2023-09-06T18:54:39.264 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.265 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:RYEZ13QtbXJqx9VanMv5lwSwvYo= 2023-09-06T18:54:39.265 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'T5i766ZVJJ3eXXOUsc6lFA==', 'Content-Length': '3010173', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:RYEZ13QtbXJqx9VanMv5lwSwvYo='} 2023-09-06T18:54:39.265 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.265 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.265 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.265 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.266 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.266 INFO:teuthology.orchestra.run.smithi099.stderr:wH4O4G0cUa3RVZTZL8Bx8g== 2023-09-06T18:54:39.266 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.266 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.267 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/74 2023-09-06T18:54:39.267 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.267 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:oXpzU2EWQ1vcOuvfW97TE3ZNzJg= 2023-09-06T18:54:39.267 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'wH4O4G0cUa3RVZTZL8Bx8g==', 'Content-Length': '3009021', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:oXpzU2EWQ1vcOuvfW97TE3ZNzJg='} 2023-09-06T18:54:39.267 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.268 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.268 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.268 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.268 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.268 INFO:teuthology.orchestra.run.smithi099.stderr:8B3cc/t8TpRZF5cCer6dEQ== 2023-09-06T18:54:39.268 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.269 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.269 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/77 2023-09-06T18:54:39.269 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.269 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VWVCrMVG65GwMoPOxodu30Gx5IY= 2023-09-06T18:54:39.269 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '8B3cc/t8TpRZF5cCer6dEQ==', 'Content-Length': '3008801', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VWVCrMVG65GwMoPOxodu30Gx5IY='} 2023-09-06T18:54:39.270 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.270 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.270 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.270 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.271 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.271 INFO:teuthology.orchestra.run.smithi099.stderr:ACjFwH5NGlKEdPWFDs0Uug== 2023-09-06T18:54:39.271 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.271 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.271 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/70 2023-09-06T18:54:39.271 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.272 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Gvl3cUJ2iRK2IcsrZZrixV76LeQ= 2023-09-06T18:54:39.272 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ACjFwH5NGlKEdPWFDs0Uug==', 'Content-Length': '3011260', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Gvl3cUJ2iRK2IcsrZZrixV76LeQ='} 2023-09-06T18:54:39.272 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.272 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.272 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.273 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.273 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.273 INFO:teuthology.orchestra.run.smithi099.stderr:ZTBtXbDjEjBhg8rpL9GNdw== 2023-09-06T18:54:39.273 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.273 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.273 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/91 2023-09-06T18:54:39.274 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.274 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:nQP1DEadb8eksNgyNVzpg8Nd0as= 2023-09-06T18:54:39.274 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ZTBtXbDjEjBhg8rpL9GNdw==', 'Content-Length': '3011579', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:nQP1DEadb8eksNgyNVzpg8Nd0as='} 2023-09-06T18:54:39.274 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.274 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.275 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.275 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.275 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.275 INFO:teuthology.orchestra.run.smithi099.stderr:TpxZegmN2A+sJ3sHtNR34g== 2023-09-06T18:54:39.275 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.275 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.276 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/81 2023-09-06T18:54:39.276 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.276 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6EPFOpoyygYmIMj+Rug1f0pP8YM= 2023-09-06T18:54:39.276 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TpxZegmN2A+sJ3sHtNR34g==', 'Content-Length': '3011797', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6EPFOpoyygYmIMj+Rug1f0pP8YM='} 2023-09-06T18:54:39.276 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.277 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.277 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.277 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.277 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.277 INFO:teuthology.orchestra.run.smithi099.stderr:6Mq+4Y9CKsucB1afANko4g== 2023-09-06T18:54:39.278 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.278 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.278 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/90 2023-09-06T18:54:39.278 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.278 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1hcMZJN2zAi4C/ciWwcDP+nO928= 2023-09-06T18:54:39.278 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '6Mq+4Y9CKsucB1afANko4g==', 'Content-Length': '3013824', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1hcMZJN2zAi4C/ciWwcDP+nO928='} 2023-09-06T18:54:39.279 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.279 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.279 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.279 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.279 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.280 INFO:teuthology.orchestra.run.smithi099.stderr:GJkR3zTdK3YwvIGvfcQsJA== 2023-09-06T18:54:39.280 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.280 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.280 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/94 2023-09-06T18:54:39.281 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.281 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1TCHMs8abxmMC5byV+PH4dfiIn8= 2023-09-06T18:54:39.281 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GJkR3zTdK3YwvIGvfcQsJA==', 'Content-Length': '3011083', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1TCHMs8abxmMC5byV+PH4dfiIn8='} 2023-09-06T18:54:39.281 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.281 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.281 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.282 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.282 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.282 INFO:teuthology.orchestra.run.smithi099.stderr:1Wi/eLV38vm0BZtxAI2RNA== 2023-09-06T18:54:39.282 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/97 2023-09-06T18:54:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:dNHETCwYqsClR6tlXTHGA3vYsm8= 2023-09-06T18:54:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1Wi/eLV38vm0BZtxAI2RNA==', 'Content-Length': '3011110', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:dNHETCwYqsClR6tlXTHGA3vYsm8='} 2023-09-06T18:54:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.285 INFO:teuthology.orchestra.run.smithi099.stderr:bzcWCe4Zs+Wz9mmNWMtfMA== 2023-09-06T18:54:39.285 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.285 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.285 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/86 2023-09-06T18:54:39.285 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:jytVWq34l+SSBNBapWZMC66VzHY= 2023-09-06T18:54:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'bzcWCe4Zs+Wz9mmNWMtfMA==', 'Content-Length': '3011876', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:jytVWq34l+SSBNBapWZMC66VzHY='} 2023-09-06T18:54:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.287 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.287 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.287 INFO:teuthology.orchestra.run.smithi099.stderr:LInDI/4s7vtCmmUdl2GZgQ== 2023-09-06T18:54:39.287 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.287 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.288 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/78 2023-09-06T18:54:39.288 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.288 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:wTYx6jGw8NbNYPV0mbzgPssE+PE= 2023-09-06T18:54:39.288 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'LInDI/4s7vtCmmUdl2GZgQ==', 'Content-Length': '3012430', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:wTYx6jGw8NbNYPV0mbzgPssE+PE='} 2023-09-06T18:54:39.288 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.289 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.289 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.289 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.289 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.289 INFO:teuthology.orchestra.run.smithi099.stderr:3FruUTzNJSEDHJpOgQE21Q== 2023-09-06T18:54:39.290 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.290 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.290 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/98 2023-09-06T18:54:39.290 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.290 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:mNSz2fBFJ+HQyMBbfHNdv778AeE= 2023-09-06T18:54:39.291 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FruUTzNJSEDHJpOgQE21Q==', 'Content-Length': '3011202', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:mNSz2fBFJ+HQyMBbfHNdv778AeE='} 2023-09-06T18:54:39.291 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.291 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.291 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.291 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.292 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.292 INFO:teuthology.orchestra.run.smithi099.stderr:onB9dpXuID5Leh9ic6perg== 2023-09-06T18:54:39.292 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.292 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.292 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/71 2023-09-06T18:54:39.293 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.293 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Nu4fqXnsX73xXTrJxiVFnc1DqK8= 2023-09-06T18:54:39.293 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'onB9dpXuID5Leh9ic6perg==', 'Content-Length': '3010257', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Nu4fqXnsX73xXTrJxiVFnc1DqK8='} 2023-09-06T18:54:39.293 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.293 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.294 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.294 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.294 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.294 INFO:teuthology.orchestra.run.smithi099.stderr:4lhMsR4INMw5LUlW+fjijg== 2023-09-06T18:54:39.294 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.295 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.295 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/89 2023-09-06T18:54:39.295 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.295 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:vnxVSmpqWU4SKgYVVv2CZeWLgTU= 2023-09-06T18:54:39.295 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '4lhMsR4INMw5LUlW+fjijg==', 'Content-Length': '3008833', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:vnxVSmpqWU4SKgYVVv2CZeWLgTU='} 2023-09-06T18:54:39.296 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.296 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.296 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.296 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.296 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.297 INFO:teuthology.orchestra.run.smithi099.stderr:IlteDkzUvp19xkYkWXYCEw== 2023-09-06T18:54:39.297 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.297 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.297 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/95 2023-09-06T18:54:39.297 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.297 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:qU8kqZa6Kd3gi2mTN1yByHiSPhs= 2023-09-06T18:54:39.298 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'IlteDkzUvp19xkYkWXYCEw==', 'Content-Length': '3010755', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:qU8kqZa6Kd3gi2mTN1yByHiSPhs='} 2023-09-06T18:54:39.298 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.298 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.298 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.298 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.299 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.299 INFO:teuthology.orchestra.run.smithi099.stderr:58/syNvnMEysIcHSAd8NBg== 2023-09-06T18:54:39.299 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.299 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.299 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/72 2023-09-06T18:54:39.299 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.300 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:34itloabOLBXXfokpjOSbKGYcPY= 2023-09-06T18:54:39.300 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '58/syNvnMEysIcHSAd8NBg==', 'Content-Length': '3012528', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:34itloabOLBXXfokpjOSbKGYcPY='} 2023-09-06T18:54:39.300 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.300 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.300 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.301 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.301 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.301 INFO:teuthology.orchestra.run.smithi099.stderr:az8m3gLwA3rfuOuGoIUlIg== 2023-09-06T18:54:39.301 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.301 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.302 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/79 2023-09-06T18:54:39.302 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.302 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:g3Ny3rsaOzsW2WwvyD68FiQb99s= 2023-09-06T18:54:39.302 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'az8m3gLwA3rfuOuGoIUlIg==', 'Content-Length': '3009702', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:g3Ny3rsaOzsW2WwvyD68FiQb99s='} 2023-09-06T18:54:39.302 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.303 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.303 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.303 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.303 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.303 INFO:teuthology.orchestra.run.smithi099.stderr:hJWNhlOi09qLpvsl0uu7Cg== 2023-09-06T18:54:39.303 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.304 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.304 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/99 2023-09-06T18:54:39.304 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.304 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Atvf3JbMyzqzFA+QlmjLbhjVYnk= 2023-09-06T18:54:39.304 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hJWNhlOi09qLpvsl0uu7Cg==', 'Content-Length': '3010645', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Atvf3JbMyzqzFA+QlmjLbhjVYnk='} 2023-09-06T18:54:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:LInDI/4s7vtCmmUdl2GZgQ== 2023-09-06T18:54:39.306 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.306 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.306 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/78 2023-09-06T18:54:39.306 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.306 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:wTYx6jGw8NbNYPV0mbzgPssE+PE= 2023-09-06T18:54:39.307 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'LInDI/4s7vtCmmUdl2GZgQ==', 'Content-Length': '3012430', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:wTYx6jGw8NbNYPV0mbzgPssE+PE='} 2023-09-06T18:54:39.307 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.307 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.307 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.307 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.307 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.308 INFO:teuthology.orchestra.run.smithi099.stderr:ivwZ7Vdh/5AezzspgtVxkA== 2023-09-06T18:54:39.308 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.308 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.308 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/73 2023-09-06T18:54:39.308 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.308 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:rAgrUjCY/2UU1peWmmZ9tsZx824= 2023-09-06T18:54:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ivwZ7Vdh/5AezzspgtVxkA==', 'Content-Length': '3008097', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:rAgrUjCY/2UU1peWmmZ9tsZx824='} 2023-09-06T18:54:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.310 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.310 INFO:teuthology.orchestra.run.smithi099.stderr:4lhMsR4INMw5LUlW+fjijg== 2023-09-06T18:54:39.310 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.310 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.310 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/89 2023-09-06T18:54:39.311 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.311 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:vnxVSmpqWU4SKgYVVv2CZeWLgTU= 2023-09-06T18:54:39.311 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '4lhMsR4INMw5LUlW+fjijg==', 'Content-Length': '3008833', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:vnxVSmpqWU4SKgYVVv2CZeWLgTU='} 2023-09-06T18:54:39.311 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.311 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.312 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.312 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.312 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.312 INFO:teuthology.orchestra.run.smithi099.stderr:O5q5DUvpDjeTWx2UQS0b/w== 2023-09-06T18:54:39.312 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.313 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.313 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/76 2023-09-06T18:54:39.313 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.313 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:klKXT0O66VTwU/oiBA1v0jgqpVU= 2023-09-06T18:54:39.313 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'O5q5DUvpDjeTWx2UQS0b/w==', 'Content-Length': '3012500', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:klKXT0O66VTwU/oiBA1v0jgqpVU='} 2023-09-06T18:54:39.314 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.314 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.314 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.314 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.314 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.315 INFO:teuthology.orchestra.run.smithi099.stderr:H/7PVRaBhaiTgBGwsocA9g== 2023-09-06T18:54:39.315 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.315 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.315 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/66 2023-09-06T18:54:39.315 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.316 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:36FUK1BTY8M1HxewVhurZeS63eU= 2023-09-06T18:54:39.316 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'H/7PVRaBhaiTgBGwsocA9g==', 'Content-Length': '3010803', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:36FUK1BTY8M1HxewVhurZeS63eU='} 2023-09-06T18:54:39.316 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.316 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.316 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.317 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.317 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.317 INFO:teuthology.orchestra.run.smithi099.stderr:Pynp8NrbQabMkK3jYh9YCQ== 2023-09-06T18:54:39.317 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.317 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.318 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/84 2023-09-06T18:54:39.318 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.318 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:DWb43RjPnXnEfLGJ6QXz3oIUtCk= 2023-09-06T18:54:39.318 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Pynp8NrbQabMkK3jYh9YCQ==', 'Content-Length': '3010540', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:DWb43RjPnXnEfLGJ6QXz3oIUtCk='} 2023-09-06T18:54:39.319 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.319 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.319 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.319 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.319 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.320 INFO:teuthology.orchestra.run.smithi099.stderr:1Wi/eLV38vm0BZtxAI2RNA== 2023-09-06T18:54:39.320 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.320 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.320 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/97 2023-09-06T18:54:39.321 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.321 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:dNHETCwYqsClR6tlXTHGA3vYsm8= 2023-09-06T18:54:39.321 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1Wi/eLV38vm0BZtxAI2RNA==', 'Content-Length': '3011110', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:dNHETCwYqsClR6tlXTHGA3vYsm8='} 2023-09-06T18:54:39.321 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.321 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.321 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.322 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.322 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.322 INFO:teuthology.orchestra.run.smithi099.stderr:C/iQp/RnJcz00qfhYKVejg== 2023-09-06T18:54:39.322 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.322 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.323 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/83 2023-09-06T18:54:39.323 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.323 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VBQiPvpgkgY0eGav2GjAQBMnjks= 2023-09-06T18:54:39.323 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C/iQp/RnJcz00qfhYKVejg==', 'Content-Length': '3008624', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VBQiPvpgkgY0eGav2GjAQBMnjks='} 2023-09-06T18:54:39.323 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.324 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.324 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.324 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.324 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.324 INFO:teuthology.orchestra.run.smithi099.stderr:C6D+oxut6krwvUPn9E6aqQ== 2023-09-06T18:54:39.324 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.325 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.325 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/68 2023-09-06T18:54:39.325 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.325 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:CtTT22ewDAnC9YahPSmJI1/2Xt0= 2023-09-06T18:54:39.325 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C6D+oxut6krwvUPn9E6aqQ==', 'Content-Length': '3009821', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:CtTT22ewDAnC9YahPSmJI1/2Xt0='} 2023-09-06T18:54:39.326 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.326 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.326 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.326 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.326 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.326 INFO:teuthology.orchestra.run.smithi099.stderr:bzcWCe4Zs+Wz9mmNWMtfMA== 2023-09-06T18:54:39.327 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.327 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.327 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/86 2023-09-06T18:54:39.327 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.327 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:jytVWq34l+SSBNBapWZMC66VzHY= 2023-09-06T18:54:39.328 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'bzcWCe4Zs+Wz9mmNWMtfMA==', 'Content-Length': '3011876', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:jytVWq34l+SSBNBapWZMC66VzHY='} 2023-09-06T18:54:39.328 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.328 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.328 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.328 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.329 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.329 INFO:teuthology.orchestra.run.smithi099.stderr:sTGY538hYoLTXfpqTlI66Q== 2023-09-06T18:54:39.329 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.329 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.329 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/64 2023-09-06T18:54:39.330 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.330 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:kPQ7Nw0ABJAdPoKDwfRqYz6cZE0= 2023-09-06T18:54:39.330 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'sTGY538hYoLTXfpqTlI66Q==', 'Content-Length': '3012286', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:kPQ7Nw0ABJAdPoKDwfRqYz6cZE0='} 2023-09-06T18:54:39.330 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.331 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.331 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.331 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.331 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.331 INFO:teuthology.orchestra.run.smithi099.stderr:L+CnbtRBIsmcKjWa1bgqAQ== 2023-09-06T18:54:39.331 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.332 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.332 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/62 2023-09-06T18:54:39.332 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.332 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:YvmJ7d/T+PZLDfTssCi26WRDB6U= 2023-09-06T18:54:39.332 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'L+CnbtRBIsmcKjWa1bgqAQ==', 'Content-Length': '3012829', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:YvmJ7d/T+PZLDfTssCi26WRDB6U='} 2023-09-06T18:54:39.333 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.333 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.333 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.333 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.333 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.333 INFO:teuthology.orchestra.run.smithi099.stderr:TpxZegmN2A+sJ3sHtNR34g== 2023-09-06T18:54:39.334 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.334 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.334 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/81 2023-09-06T18:54:39.334 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.334 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6EPFOpoyygYmIMj+Rug1f0pP8YM= 2023-09-06T18:54:39.334 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TpxZegmN2A+sJ3sHtNR34g==', 'Content-Length': '3011797', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6EPFOpoyygYmIMj+Rug1f0pP8YM='} 2023-09-06T18:54:39.335 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.335 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.335 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.335 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.335 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.336 INFO:teuthology.orchestra.run.smithi099.stderr:8B3cc/t8TpRZF5cCer6dEQ== 2023-09-06T18:54:39.336 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.336 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.336 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/77 2023-09-06T18:54:39.336 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.336 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VWVCrMVG65GwMoPOxodu30Gx5IY= 2023-09-06T18:54:39.337 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '8B3cc/t8TpRZF5cCer6dEQ==', 'Content-Length': '3008801', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VWVCrMVG65GwMoPOxodu30Gx5IY='} 2023-09-06T18:54:39.337 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.337 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.337 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.337 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.338 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.338 INFO:teuthology.orchestra.run.smithi099.stderr:6Mq+4Y9CKsucB1afANko4g== 2023-09-06T18:54:39.338 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.338 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.338 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/90 2023-09-06T18:54:39.338 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.339 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1hcMZJN2zAi4C/ciWwcDP+nO928= 2023-09-06T18:54:39.339 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '6Mq+4Y9CKsucB1afANko4g==', 'Content-Length': '3013824', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1hcMZJN2zAi4C/ciWwcDP+nO928='} 2023-09-06T18:54:39.339 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.339 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.339 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.340 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.340 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.340 INFO:teuthology.orchestra.run.smithi099.stderr:IlteDkzUvp19xkYkWXYCEw== 2023-09-06T18:54:39.340 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.340 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.341 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/95 2023-09-06T18:54:39.341 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.341 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:qU8kqZa6Kd3gi2mTN1yByHiSPhs= 2023-09-06T18:54:39.341 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'IlteDkzUvp19xkYkWXYCEw==', 'Content-Length': '3010755', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:qU8kqZa6Kd3gi2mTN1yByHiSPhs='} 2023-09-06T18:54:39.341 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.342 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.342 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.342 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.342 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.342 INFO:teuthology.orchestra.run.smithi099.stderr:58/syNvnMEysIcHSAd8NBg== 2023-09-06T18:54:39.343 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.343 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.343 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/72 2023-09-06T18:54:39.343 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.343 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:34itloabOLBXXfokpjOSbKGYcPY= 2023-09-06T18:54:39.344 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '58/syNvnMEysIcHSAd8NBg==', 'Content-Length': '3012528', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:34itloabOLBXXfokpjOSbKGYcPY='} 2023-09-06T18:54:39.344 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.344 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.344 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.344 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.344 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.345 INFO:teuthology.orchestra.run.smithi099.stderr:SCQrX/nmwhId+jfW+hbFKA== 2023-09-06T18:54:39.345 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.345 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.345 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/82 2023-09-06T18:54:39.345 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.346 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:0g54EVFLjN5wdXtHGIks2sEB4/o= 2023-09-06T18:54:39.346 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'SCQrX/nmwhId+jfW+hbFKA==', 'Content-Length': '3013892', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:0g54EVFLjN5wdXtHGIks2sEB4/o='} 2023-09-06T18:54:39.346 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.346 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.346 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:MILNOxKsGQkBzSCgDskNTA== 2023-09-06T18:54:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/80 2023-09-06T18:54:39.348 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.348 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:5sx0j0PgApPbvZFYmcsddlB3Dig= 2023-09-06T18:54:39.348 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MILNOxKsGQkBzSCgDskNTA==', 'Content-Length': '3009370', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:5sx0j0PgApPbvZFYmcsddlB3Dig='} 2023-09-06T18:54:39.348 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.348 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.348 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.349 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.349 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.349 INFO:teuthology.orchestra.run.smithi099.stderr:13KPoJjsWZmb48Fg0K6R/g== 2023-09-06T18:54:39.349 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.349 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.349 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/75 2023-09-06T18:54:39.350 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.350 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:8wq87cdlwIhv6AMzvkexpnLie1o= 2023-09-06T18:54:39.350 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '13KPoJjsWZmb48Fg0K6R/g==', 'Content-Length': '3010936', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:8wq87cdlwIhv6AMzvkexpnLie1o='} 2023-09-06T18:54:39.350 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.350 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.351 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.351 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.351 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.351 INFO:teuthology.orchestra.run.smithi099.stderr:vUxwqUKsZaB1t8UY3dtHOg== 2023-09-06T18:54:39.351 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.351 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.352 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/87 2023-09-06T18:54:39.352 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.352 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:EBRZ6AH9z0Bo+QZREF5CA5CZ9XA= 2023-09-06T18:54:39.352 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'vUxwqUKsZaB1t8UY3dtHOg==', 'Content-Length': '3009715', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:EBRZ6AH9z0Bo+QZREF5CA5CZ9XA='} 2023-09-06T18:54:39.352 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.352 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:d1WW15syDrKs3UdrMlkfWg== 2023-09-06T18:54:39.354 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.354 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.354 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/92 2023-09-06T18:54:39.354 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.354 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:4bsf40rLqAbfE0v5i1qQUo03ox4= 2023-09-06T18:54:39.355 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'd1WW15syDrKs3UdrMlkfWg==', 'Content-Length': '3010384', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:4bsf40rLqAbfE0v5i1qQUo03ox4='} 2023-09-06T18:54:39.355 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.355 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.355 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.355 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:d1WW15syDrKs3UdrMlkfWg== 2023-09-06T18:54:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/92 2023-09-06T18:54:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:4bsf40rLqAbfE0v5i1qQUo03ox4= 2023-09-06T18:54:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'd1WW15syDrKs3UdrMlkfWg==', 'Content-Length': '3010384', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:4bsf40rLqAbfE0v5i1qQUo03ox4='} 2023-09-06T18:54:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:MwPOSAa17PNqNyNds46rPQ== 2023-09-06T18:54:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/88 2023-09-06T18:54:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:PYSmmu8GGwgwE32n73PXNZEqkko= 2023-09-06T18:54:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MwPOSAa17PNqNyNds46rPQ==', 'Content-Length': '3010515', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:PYSmmu8GGwgwE32n73PXNZEqkko='} 2023-09-06T18:54:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:TloLB9lPBB2/TtlB2Gy1wA== 2023-09-06T18:54:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/69 2023-09-06T18:54:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6qcEOM3G1/zk11ieYYG76Z/6oiA= 2023-09-06T18:54:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TloLB9lPBB2/TtlB2Gy1wA==', 'Content-Length': '3009018', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6qcEOM3G1/zk11ieYYG76Z/6oiA='} 2023-09-06T18:54:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:ZTBtXbDjEjBhg8rpL9GNdw== 2023-09-06T18:54:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/91 2023-09-06T18:54:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.364 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:nQP1DEadb8eksNgyNVzpg8Nd0as= 2023-09-06T18:54:39.364 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ZTBtXbDjEjBhg8rpL9GNdw==', 'Content-Length': '3011579', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:nQP1DEadb8eksNgyNVzpg8Nd0as='} 2023-09-06T18:54:39.364 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.364 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.364 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.364 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.365 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.365 INFO:teuthology.orchestra.run.smithi099.stderr:ACjFwH5NGlKEdPWFDs0Uug== 2023-09-06T18:54:39.365 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.365 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.365 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/70 2023-09-06T18:54:39.365 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.366 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Gvl3cUJ2iRK2IcsrZZrixV76LeQ= 2023-09-06T18:54:39.366 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ACjFwH5NGlKEdPWFDs0Uug==', 'Content-Length': '3011260', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Gvl3cUJ2iRK2IcsrZZrixV76LeQ='} 2023-09-06T18:54:39.366 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.366 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.367 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.367 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.367 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.367 INFO:teuthology.orchestra.run.smithi099.stderr:ivwZ7Vdh/5AezzspgtVxkA== 2023-09-06T18:54:39.367 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.368 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.368 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/73 2023-09-06T18:54:39.368 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.368 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:rAgrUjCY/2UU1peWmmZ9tsZx824= 2023-09-06T18:54:39.368 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ivwZ7Vdh/5AezzspgtVxkA==', 'Content-Length': '3008097', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:rAgrUjCY/2UU1peWmmZ9tsZx824='} 2023-09-06T18:54:39.368 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.369 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.369 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.369 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.369 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.369 INFO:teuthology.orchestra.run.smithi099.stderr:JY4+ecb2EmwJv//Rf3oChw== 2023-09-06T18:54:39.369 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.370 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.370 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/65 2023-09-06T18:54:39.370 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.370 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:e7Brq42o6Lust4KBxlCx+CThsJ0= 2023-09-06T18:54:39.370 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'JY4+ecb2EmwJv//Rf3oChw==', 'Content-Length': '3010799', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:e7Brq42o6Lust4KBxlCx+CThsJ0='} 2023-09-06T18:54:39.371 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.371 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.371 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.371 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.371 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.371 INFO:teuthology.orchestra.run.smithi099.stderr:ApVRZRTGuJLDF4R7yT+TnA== 2023-09-06T18:54:39.372 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.372 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.372 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/93 2023-09-06T18:54:39.372 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.372 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:T0yfDwfwBTB5XlDGzeKSZspocV0= 2023-09-06T18:54:39.373 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ApVRZRTGuJLDF4R7yT+TnA==', 'Content-Length': '3008841', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:T0yfDwfwBTB5XlDGzeKSZspocV0='} 2023-09-06T18:54:39.373 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.373 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.373 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.373 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.373 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.374 INFO:teuthology.orchestra.run.smithi099.stderr:wH4O4G0cUa3RVZTZL8Bx8g== 2023-09-06T18:54:39.374 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.374 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.374 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/74 2023-09-06T18:54:39.374 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.375 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:oXpzU2EWQ1vcOuvfW97TE3ZNzJg= 2023-09-06T18:54:39.375 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'wH4O4G0cUa3RVZTZL8Bx8g==', 'Content-Length': '3009021', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:oXpzU2EWQ1vcOuvfW97TE3ZNzJg='} 2023-09-06T18:54:39.375 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.375 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.375 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.375 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.376 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.376 INFO:teuthology.orchestra.run.smithi099.stderr:OkRpNuHoekxqIV5d/AEmQw== 2023-09-06T18:54:39.376 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.376 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.376 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/96 2023-09-06T18:54:39.377 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.377 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VNYADlT08pfUnJaj4kH1qEp0w0M= 2023-09-06T18:54:39.377 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'OkRpNuHoekxqIV5d/AEmQw==', 'Content-Length': '3010153', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VNYADlT08pfUnJaj4kH1qEp0w0M='} 2023-09-06T18:54:39.377 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.377 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.378 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.378 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.378 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.378 INFO:teuthology.orchestra.run.smithi099.stderr:1Wi/eLV38vm0BZtxAI2RNA== 2023-09-06T18:54:39.378 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.379 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.379 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/97 2023-09-06T18:54:39.379 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.379 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:dNHETCwYqsClR6tlXTHGA3vYsm8= 2023-09-06T18:54:39.379 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1Wi/eLV38vm0BZtxAI2RNA==', 'Content-Length': '3011110', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:dNHETCwYqsClR6tlXTHGA3vYsm8='} 2023-09-06T18:54:39.380 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.380 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.380 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.380 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.380 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:FowqMMckz0VWUhuaNxe2nQ== 2023-09-06T18:54:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/60 2023-09-06T18:54:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.382 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1HpIPoejfKGUqWjAYirHX5A9szI= 2023-09-06T18:54:39.382 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'FowqMMckz0VWUhuaNxe2nQ==', 'Content-Length': '3011279', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1HpIPoejfKGUqWjAYirHX5A9szI='} 2023-09-06T18:54:39.382 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.382 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.382 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.382 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.383 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.383 INFO:teuthology.orchestra.run.smithi099.stderr:sTGY538hYoLTXfpqTlI66Q== 2023-09-06T18:54:39.383 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.383 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.383 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/64 2023-09-06T18:54:39.384 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.384 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:kPQ7Nw0ABJAdPoKDwfRqYz6cZE0= 2023-09-06T18:54:39.384 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'sTGY538hYoLTXfpqTlI66Q==', 'Content-Length': '3012286', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:kPQ7Nw0ABJAdPoKDwfRqYz6cZE0='} 2023-09-06T18:54:39.384 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.384 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:ZTBtXbDjEjBhg8rpL9GNdw== 2023-09-06T18:54:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/91 2023-09-06T18:54:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:nQP1DEadb8eksNgyNVzpg8Nd0as= 2023-09-06T18:54:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ZTBtXbDjEjBhg8rpL9GNdw==', 'Content-Length': '3011579', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:nQP1DEadb8eksNgyNVzpg8Nd0as='} 2023-09-06T18:54:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:dy8UZfYnVbeBuu0eEEEAdw== 2023-09-06T18:54:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/59 2023-09-06T18:54:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:FMb0hLbWXab8IMiyuTx2Sd1t5k4= 2023-09-06T18:54:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'dy8UZfYnVbeBuu0eEEEAdw==', 'Content-Length': '3013085', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:FMb0hLbWXab8IMiyuTx2Sd1t5k4='} 2023-09-06T18:54:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:6Mq+4Y9CKsucB1afANko4g== 2023-09-06T18:54:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.391 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/90 2023-09-06T18:54:39.391 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.391 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1hcMZJN2zAi4C/ciWwcDP+nO928= 2023-09-06T18:54:39.391 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '6Mq+4Y9CKsucB1afANko4g==', 'Content-Length': '3013824', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1hcMZJN2zAi4C/ciWwcDP+nO928='} 2023-09-06T18:54:39.391 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:GJkR3zTdK3YwvIGvfcQsJA== 2023-09-06T18:54:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/94 2023-09-06T18:54:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1TCHMs8abxmMC5byV+PH4dfiIn8= 2023-09-06T18:54:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GJkR3zTdK3YwvIGvfcQsJA==', 'Content-Length': '3011083', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1TCHMs8abxmMC5byV+PH4dfiIn8='} 2023-09-06T18:54:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:MwPOSAa17PNqNyNds46rPQ== 2023-09-06T18:54:39.395 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.395 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.395 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/88 2023-09-06T18:54:39.395 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.395 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:PYSmmu8GGwgwE32n73PXNZEqkko= 2023-09-06T18:54:39.395 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MwPOSAa17PNqNyNds46rPQ==', 'Content-Length': '3010515', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:PYSmmu8GGwgwE32n73PXNZEqkko='} 2023-09-06T18:54:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.397 INFO:teuthology.orchestra.run.smithi099.stderr:O5q5DUvpDjeTWx2UQS0b/w== 2023-09-06T18:54:39.397 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.397 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.397 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/76 2023-09-06T18:54:39.397 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.398 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:klKXT0O66VTwU/oiBA1v0jgqpVU= 2023-09-06T18:54:39.398 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'O5q5DUvpDjeTWx2UQS0b/w==', 'Content-Length': '3012500', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:klKXT0O66VTwU/oiBA1v0jgqpVU='} 2023-09-06T18:54:39.398 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.398 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.398 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.399 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.399 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.399 INFO:teuthology.orchestra.run.smithi099.stderr:C6D+oxut6krwvUPn9E6aqQ== 2023-09-06T18:54:39.399 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.399 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.400 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/68 2023-09-06T18:54:39.400 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.400 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:CtTT22ewDAnC9YahPSmJI1/2Xt0= 2023-09-06T18:54:39.400 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C6D+oxut6krwvUPn9E6aqQ==', 'Content-Length': '3009821', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:CtTT22ewDAnC9YahPSmJI1/2Xt0='} 2023-09-06T18:54:39.400 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.400 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.401 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.401 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.401 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.401 INFO:teuthology.orchestra.run.smithi099.stderr:ACjFwH5NGlKEdPWFDs0Uug== 2023-09-06T18:54:39.401 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/70 2023-09-06T18:54:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Gvl3cUJ2iRK2IcsrZZrixV76LeQ= 2023-09-06T18:54:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ACjFwH5NGlKEdPWFDs0Uug==', 'Content-Length': '3011260', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Gvl3cUJ2iRK2IcsrZZrixV76LeQ='} 2023-09-06T18:54:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.404 INFO:teuthology.orchestra.run.smithi099.stderr:ZTBtXbDjEjBhg8rpL9GNdw== 2023-09-06T18:54:39.404 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.404 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.404 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/91 2023-09-06T18:54:39.404 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.404 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:nQP1DEadb8eksNgyNVzpg8Nd0as= 2023-09-06T18:54:39.405 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ZTBtXbDjEjBhg8rpL9GNdw==', 'Content-Length': '3011579', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:nQP1DEadb8eksNgyNVzpg8Nd0as='} 2023-09-06T18:54:39.405 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.405 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.405 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.406 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.406 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.406 INFO:teuthology.orchestra.run.smithi099.stderr:onB9dpXuID5Leh9ic6perg== 2023-09-06T18:54:39.406 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.406 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/71 2023-09-06T18:54:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Nu4fqXnsX73xXTrJxiVFnc1DqK8= 2023-09-06T18:54:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'onB9dpXuID5Leh9ic6perg==', 'Content-Length': '3010257', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Nu4fqXnsX73xXTrJxiVFnc1DqK8='} 2023-09-06T18:54:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:wH4O4G0cUa3RVZTZL8Bx8g== 2023-09-06T18:54:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.409 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.409 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/74 2023-09-06T18:54:39.409 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.409 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:oXpzU2EWQ1vcOuvfW97TE3ZNzJg= 2023-09-06T18:54:39.409 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'wH4O4G0cUa3RVZTZL8Bx8g==', 'Content-Length': '3009021', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:oXpzU2EWQ1vcOuvfW97TE3ZNzJg='} 2023-09-06T18:54:39.410 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.410 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.410 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.410 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.410 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.411 INFO:teuthology.orchestra.run.smithi099.stderr:13KPoJjsWZmb48Fg0K6R/g== 2023-09-06T18:54:39.411 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.411 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.411 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/75 2023-09-06T18:54:39.411 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.412 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:8wq87cdlwIhv6AMzvkexpnLie1o= 2023-09-06T18:54:39.412 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '13KPoJjsWZmb48Fg0K6R/g==', 'Content-Length': '3010936', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:8wq87cdlwIhv6AMzvkexpnLie1o='} 2023-09-06T18:54:39.412 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.412 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.412 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.413 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.413 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.413 INFO:teuthology.orchestra.run.smithi099.stderr:CExctBxSvJfXGKs/K/T9MQ== 2023-09-06T18:54:39.413 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.413 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.413 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/67 2023-09-06T18:54:39.414 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.414 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Kg+8r2tVv0ClHQIwVtGnpmy4f0A= 2023-09-06T18:54:39.414 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'CExctBxSvJfXGKs/K/T9MQ==', 'Content-Length': '3011407', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Kg+8r2tVv0ClHQIwVtGnpmy4f0A='} 2023-09-06T18:54:39.414 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.414 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.415 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.415 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.415 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.415 INFO:teuthology.orchestra.run.smithi099.stderr:az8m3gLwA3rfuOuGoIUlIg== 2023-09-06T18:54:39.416 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.416 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.416 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/79 2023-09-06T18:54:39.416 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.416 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:g3Ny3rsaOzsW2WwvyD68FiQb99s= 2023-09-06T18:54:39.417 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'az8m3gLwA3rfuOuGoIUlIg==', 'Content-Length': '3009702', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:g3Ny3rsaOzsW2WwvyD68FiQb99s='} 2023-09-06T18:54:39.417 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.417 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.417 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.417 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.417 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.418 INFO:teuthology.orchestra.run.smithi099.stderr:sTGY538hYoLTXfpqTlI66Q== 2023-09-06T18:54:39.418 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.418 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.418 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/64 2023-09-06T18:54:39.419 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.419 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:kPQ7Nw0ABJAdPoKDwfRqYz6cZE0= 2023-09-06T18:54:39.419 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'sTGY538hYoLTXfpqTlI66Q==', 'Content-Length': '3012286', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:kPQ7Nw0ABJAdPoKDwfRqYz6cZE0='} 2023-09-06T18:54:39.419 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.419 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.419 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.420 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.420 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.420 INFO:teuthology.orchestra.run.smithi099.stderr:8B3cc/t8TpRZF5cCer6dEQ== 2023-09-06T18:54:39.420 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.421 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.421 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/77 2023-09-06T18:54:39.421 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.421 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VWVCrMVG65GwMoPOxodu30Gx5IY= 2023-09-06T18:54:39.421 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '8B3cc/t8TpRZF5cCer6dEQ==', 'Content-Length': '3008801', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VWVCrMVG65GwMoPOxodu30Gx5IY='} 2023-09-06T18:54:39.421 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.422 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.422 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.422 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.422 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.423 INFO:teuthology.orchestra.run.smithi099.stderr:OkRpNuHoekxqIV5d/AEmQw== 2023-09-06T18:54:39.423 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.423 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.423 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/96 2023-09-06T18:54:39.423 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.423 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VNYADlT08pfUnJaj4kH1qEp0w0M= 2023-09-06T18:54:39.424 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'OkRpNuHoekxqIV5d/AEmQw==', 'Content-Length': '3010153', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VNYADlT08pfUnJaj4kH1qEp0w0M='} 2023-09-06T18:54:39.424 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.424 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.424 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.424 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.425 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.425 INFO:teuthology.orchestra.run.smithi099.stderr:3FruUTzNJSEDHJpOgQE21Q== 2023-09-06T18:54:39.425 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.425 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.425 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/98 2023-09-06T18:54:39.426 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.426 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:mNSz2fBFJ+HQyMBbfHNdv778AeE= 2023-09-06T18:54:39.426 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FruUTzNJSEDHJpOgQE21Q==', 'Content-Length': '3011202', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:mNSz2fBFJ+HQyMBbfHNdv778AeE='} 2023-09-06T18:54:39.426 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.426 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.427 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.427 INFO:teuthology.orchestra.run.smithi099.stderr:LInDI/4s7vtCmmUdl2GZgQ== 2023-09-06T18:54:39.427 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.427 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.427 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/78 2023-09-06T18:54:39.428 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.428 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:wTYx6jGw8NbNYPV0mbzgPssE+PE= 2023-09-06T18:54:39.428 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'LInDI/4s7vtCmmUdl2GZgQ==', 'Content-Length': '3012430', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:wTYx6jGw8NbNYPV0mbzgPssE+PE='} 2023-09-06T18:54:39.428 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.428 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.429 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.429 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.429 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.429 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.429 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.429 INFO:teuthology.orchestra.run.smithi099.stderr:ivwZ7Vdh/5AezzspgtVxkA== 2023-09-06T18:54:39.430 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.430 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.430 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/73 2023-09-06T18:54:39.430 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.430 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:rAgrUjCY/2UU1peWmmZ9tsZx824= 2023-09-06T18:54:39.431 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ivwZ7Vdh/5AezzspgtVxkA==', 'Content-Length': '3008097', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:rAgrUjCY/2UU1peWmmZ9tsZx824='} 2023-09-06T18:54:39.431 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.431 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.431 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.432 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.432 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.432 INFO:teuthology.orchestra.run.smithi099.stderr:58/syNvnMEysIcHSAd8NBg== 2023-09-06T18:54:39.432 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.432 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.433 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/72 2023-09-06T18:54:39.433 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.433 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:34itloabOLBXXfokpjOSbKGYcPY= 2023-09-06T18:54:39.433 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '58/syNvnMEysIcHSAd8NBg==', 'Content-Length': '3012528', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:34itloabOLBXXfokpjOSbKGYcPY='} 2023-09-06T18:54:39.433 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.433 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.434 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.434 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.434 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.434 INFO:teuthology.orchestra.run.smithi099.stderr:bzcWCe4Zs+Wz9mmNWMtfMA== 2023-09-06T18:54:39.434 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.435 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.435 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/86 2023-09-06T18:54:39.435 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.435 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:jytVWq34l+SSBNBapWZMC66VzHY= 2023-09-06T18:54:39.436 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'bzcWCe4Zs+Wz9mmNWMtfMA==', 'Content-Length': '3011876', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:jytVWq34l+SSBNBapWZMC66VzHY='} 2023-09-06T18:54:39.436 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.436 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.436 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.436 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:SCQrX/nmwhId+jfW+hbFKA== 2023-09-06T18:54:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/82 2023-09-06T18:54:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:0g54EVFLjN5wdXtHGIks2sEB4/o= 2023-09-06T18:54:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'SCQrX/nmwhId+jfW+hbFKA==', 'Content-Length': '3013892', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:0g54EVFLjN5wdXtHGIks2sEB4/o='} 2023-09-06T18:54:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:8B3cc/t8TpRZF5cCer6dEQ== 2023-09-06T18:54:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/77 2023-09-06T18:54:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VWVCrMVG65GwMoPOxodu30Gx5IY= 2023-09-06T18:54:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '8B3cc/t8TpRZF5cCer6dEQ==', 'Content-Length': '3008801', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VWVCrMVG65GwMoPOxodu30Gx5IY='} 2023-09-06T18:54:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.441 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.441 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.441 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.441 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.441 INFO:teuthology.orchestra.run.smithi099.stderr:onB9dpXuID5Leh9ic6perg== 2023-09-06T18:54:39.442 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.442 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.442 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/71 2023-09-06T18:54:39.442 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.442 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Nu4fqXnsX73xXTrJxiVFnc1DqK8= 2023-09-06T18:54:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'onB9dpXuID5Leh9ic6perg==', 'Content-Length': '3010257', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Nu4fqXnsX73xXTrJxiVFnc1DqK8='} 2023-09-06T18:54:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.444 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.444 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.444 INFO:teuthology.orchestra.run.smithi099.stderr:T5i766ZVJJ3eXXOUsc6lFA== 2023-09-06T18:54:39.444 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.444 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.444 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/63 2023-09-06T18:54:39.445 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.445 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:RYEZ13QtbXJqx9VanMv5lwSwvYo= 2023-09-06T18:54:39.445 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'T5i766ZVJJ3eXXOUsc6lFA==', 'Content-Length': '3010173', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:RYEZ13QtbXJqx9VanMv5lwSwvYo='} 2023-09-06T18:54:39.445 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.445 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:MwPOSAa17PNqNyNds46rPQ== 2023-09-06T18:54:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/88 2023-09-06T18:54:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:PYSmmu8GGwgwE32n73PXNZEqkko= 2023-09-06T18:54:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MwPOSAa17PNqNyNds46rPQ==', 'Content-Length': '3010515', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:PYSmmu8GGwgwE32n73PXNZEqkko='} 2023-09-06T18:54:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:hJWNhlOi09qLpvsl0uu7Cg== 2023-09-06T18:54:39.449 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.449 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.449 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/99 2023-09-06T18:54:39.449 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.449 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Atvf3JbMyzqzFA+QlmjLbhjVYnk= 2023-09-06T18:54:39.450 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hJWNhlOi09qLpvsl0uu7Cg==', 'Content-Length': '3010645', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Atvf3JbMyzqzFA+QlmjLbhjVYnk='} 2023-09-06T18:54:39.450 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.450 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.450 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.450 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:TloLB9lPBB2/TtlB2Gy1wA== 2023-09-06T18:54:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/69 2023-09-06T18:54:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6qcEOM3G1/zk11ieYYG76Z/6oiA= 2023-09-06T18:54:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TloLB9lPBB2/TtlB2Gy1wA==', 'Content-Length': '3009018', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6qcEOM3G1/zk11ieYYG76Z/6oiA='} 2023-09-06T18:54:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.453 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.453 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.453 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.453 INFO:teuthology.orchestra.run.smithi099.stderr:3FruUTzNJSEDHJpOgQE21Q== 2023-09-06T18:54:39.453 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.454 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.454 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/98 2023-09-06T18:54:39.454 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.454 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:mNSz2fBFJ+HQyMBbfHNdv778AeE= 2023-09-06T18:54:39.454 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FruUTzNJSEDHJpOgQE21Q==', 'Content-Length': '3011202', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:mNSz2fBFJ+HQyMBbfHNdv778AeE='} 2023-09-06T18:54:39.455 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.455 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.455 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.455 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.455 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.455 INFO:teuthology.orchestra.run.smithi099.stderr:4lhMsR4INMw5LUlW+fjijg== 2023-09-06T18:54:39.456 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.456 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.456 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/89 2023-09-06T18:54:39.456 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.456 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:vnxVSmpqWU4SKgYVVv2CZeWLgTU= 2023-09-06T18:54:39.457 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '4lhMsR4INMw5LUlW+fjijg==', 'Content-Length': '3008833', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:vnxVSmpqWU4SKgYVVv2CZeWLgTU='} 2023-09-06T18:54:39.457 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.457 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.457 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.457 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.458 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.458 INFO:teuthology.orchestra.run.smithi099.stderr:JY4+ecb2EmwJv//Rf3oChw== 2023-09-06T18:54:39.458 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.458 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.458 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/65 2023-09-06T18:54:39.459 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.459 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:e7Brq42o6Lust4KBxlCx+CThsJ0= 2023-09-06T18:54:39.459 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'JY4+ecb2EmwJv//Rf3oChw==', 'Content-Length': '3010799', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:e7Brq42o6Lust4KBxlCx+CThsJ0='} 2023-09-06T18:54:39.459 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.459 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.460 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.460 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.460 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.460 INFO:teuthology.orchestra.run.smithi099.stderr:IlteDkzUvp19xkYkWXYCEw== 2023-09-06T18:54:39.460 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.460 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.461 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/95 2023-09-06T18:54:39.461 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.461 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:qU8kqZa6Kd3gi2mTN1yByHiSPhs= 2023-09-06T18:54:39.461 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'IlteDkzUvp19xkYkWXYCEw==', 'Content-Length': '3010755', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:qU8kqZa6Kd3gi2mTN1yByHiSPhs='} 2023-09-06T18:54:39.461 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.462 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.462 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.462 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.462 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.462 INFO:teuthology.orchestra.run.smithi099.stderr:TpxZegmN2A+sJ3sHtNR34g== 2023-09-06T18:54:39.463 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.463 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.463 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/81 2023-09-06T18:54:39.463 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.463 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6EPFOpoyygYmIMj+Rug1f0pP8YM= 2023-09-06T18:54:39.464 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TpxZegmN2A+sJ3sHtNR34g==', 'Content-Length': '3011797', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6EPFOpoyygYmIMj+Rug1f0pP8YM='} 2023-09-06T18:54:39.464 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.464 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.464 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.464 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.465 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.465 INFO:teuthology.orchestra.run.smithi099.stderr:FowqMMckz0VWUhuaNxe2nQ== 2023-09-06T18:54:39.465 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.465 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.465 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/60 2023-09-06T18:54:39.466 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.466 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1HpIPoejfKGUqWjAYirHX5A9szI= 2023-09-06T18:54:39.466 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'FowqMMckz0VWUhuaNxe2nQ==', 'Content-Length': '3011279', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1HpIPoejfKGUqWjAYirHX5A9szI='} 2023-09-06T18:54:39.466 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.466 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.467 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.467 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.467 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.467 INFO:teuthology.orchestra.run.smithi099.stderr:L+CnbtRBIsmcKjWa1bgqAQ== 2023-09-06T18:54:39.467 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.468 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.468 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/62 2023-09-06T18:54:39.468 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.468 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:YvmJ7d/T+PZLDfTssCi26WRDB6U= 2023-09-06T18:54:39.468 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'L+CnbtRBIsmcKjWa1bgqAQ==', 'Content-Length': '3012829', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:YvmJ7d/T+PZLDfTssCi26WRDB6U='} 2023-09-06T18:54:39.469 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.469 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.469 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.469 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.469 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.470 INFO:teuthology.orchestra.run.smithi099.stderr:vUxwqUKsZaB1t8UY3dtHOg== 2023-09-06T18:54:39.470 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.470 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.470 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/87 2023-09-06T18:54:39.470 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.471 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:EBRZ6AH9z0Bo+QZREF5CA5CZ9XA= 2023-09-06T18:54:39.471 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'vUxwqUKsZaB1t8UY3dtHOg==', 'Content-Length': '3009715', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:EBRZ6AH9z0Bo+QZREF5CA5CZ9XA='} 2023-09-06T18:54:39.471 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.471 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.471 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.472 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.472 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.472 INFO:teuthology.orchestra.run.smithi099.stderr:dy8UZfYnVbeBuu0eEEEAdw== 2023-09-06T18:54:39.472 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.472 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.473 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/59 2023-09-06T18:54:39.473 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.473 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:FMb0hLbWXab8IMiyuTx2Sd1t5k4= 2023-09-06T18:54:39.473 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'dy8UZfYnVbeBuu0eEEEAdw==', 'Content-Length': '3013085', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:FMb0hLbWXab8IMiyuTx2Sd1t5k4='} 2023-09-06T18:54:39.473 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.474 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.474 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.474 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.474 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.474 INFO:teuthology.orchestra.run.smithi099.stderr:Pynp8NrbQabMkK3jYh9YCQ== 2023-09-06T18:54:39.474 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.475 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.475 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/84 2023-09-06T18:54:39.475 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.475 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:DWb43RjPnXnEfLGJ6QXz3oIUtCk= 2023-09-06T18:54:39.475 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Pynp8NrbQabMkK3jYh9YCQ==', 'Content-Length': '3010540', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:DWb43RjPnXnEfLGJ6QXz3oIUtCk='} 2023-09-06T18:54:39.476 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.476 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.476 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.476 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.476 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.477 INFO:teuthology.orchestra.run.smithi099.stderr:C/iQp/RnJcz00qfhYKVejg== 2023-09-06T18:54:39.477 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.477 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.477 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/83 2023-09-06T18:54:39.477 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.478 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VBQiPvpgkgY0eGav2GjAQBMnjks= 2023-09-06T18:54:39.478 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C/iQp/RnJcz00qfhYKVejg==', 'Content-Length': '3008624', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VBQiPvpgkgY0eGav2GjAQBMnjks='} 2023-09-06T18:54:39.478 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.478 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.478 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.479 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.479 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.479 INFO:teuthology.orchestra.run.smithi099.stderr:MILNOxKsGQkBzSCgDskNTA== 2023-09-06T18:54:39.479 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.479 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.480 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/80 2023-09-06T18:54:39.480 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.480 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:5sx0j0PgApPbvZFYmcsddlB3Dig= 2023-09-06T18:54:39.480 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MILNOxKsGQkBzSCgDskNTA==', 'Content-Length': '3009370', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:5sx0j0PgApPbvZFYmcsddlB3Dig='} 2023-09-06T18:54:39.480 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.481 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.481 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.481 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.481 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.481 INFO:teuthology.orchestra.run.smithi099.stderr:58/syNvnMEysIcHSAd8NBg== 2023-09-06T18:54:39.482 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.482 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.482 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/72 2023-09-06T18:54:39.482 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.482 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:34itloabOLBXXfokpjOSbKGYcPY= 2023-09-06T18:54:39.483 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '58/syNvnMEysIcHSAd8NBg==', 'Content-Length': '3012528', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:34itloabOLBXXfokpjOSbKGYcPY='} 2023-09-06T18:54:39.483 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.483 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.483 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.483 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.483 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.484 INFO:teuthology.orchestra.run.smithi099.stderr:H/7PVRaBhaiTgBGwsocA9g== 2023-09-06T18:54:39.484 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.484 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.484 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/66 2023-09-06T18:54:39.484 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.485 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:36FUK1BTY8M1HxewVhurZeS63eU= 2023-09-06T18:54:39.485 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'H/7PVRaBhaiTgBGwsocA9g==', 'Content-Length': '3010803', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:36FUK1BTY8M1HxewVhurZeS63eU='} 2023-09-06T18:54:39.485 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.485 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.485 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.486 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.486 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.486 INFO:teuthology.orchestra.run.smithi099.stderr:SCQrX/nmwhId+jfW+hbFKA== 2023-09-06T18:54:39.486 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.486 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.487 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/82 2023-09-06T18:54:39.487 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.487 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:0g54EVFLjN5wdXtHGIks2sEB4/o= 2023-09-06T18:54:39.487 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'SCQrX/nmwhId+jfW+hbFKA==', 'Content-Length': '3013892', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:0g54EVFLjN5wdXtHGIks2sEB4/o='} 2023-09-06T18:54:39.487 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.488 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.488 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.488 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.488 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.488 INFO:teuthology.orchestra.run.smithi099.stderr:JY4+ecb2EmwJv//Rf3oChw== 2023-09-06T18:54:39.488 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.489 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.489 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/65 2023-09-06T18:54:39.489 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.489 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:e7Brq42o6Lust4KBxlCx+CThsJ0= 2023-09-06T18:54:39.489 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'JY4+ecb2EmwJv//Rf3oChw==', 'Content-Length': '3010799', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:e7Brq42o6Lust4KBxlCx+CThsJ0='} 2023-09-06T18:54:39.490 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.490 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.490 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.490 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.490 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.491 INFO:teuthology.orchestra.run.smithi099.stderr:ApVRZRTGuJLDF4R7yT+TnA== 2023-09-06T18:54:39.491 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.491 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.491 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/93 2023-09-06T18:54:39.491 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:T0yfDwfwBTB5XlDGzeKSZspocV0= 2023-09-06T18:54:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ApVRZRTGuJLDF4R7yT+TnA==', 'Content-Length': '3008841', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:T0yfDwfwBTB5XlDGzeKSZspocV0='} 2023-09-06T18:54:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:4lhMsR4INMw5LUlW+fjijg== 2023-09-06T18:54:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/89 2023-09-06T18:54:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:vnxVSmpqWU4SKgYVVv2CZeWLgTU= 2023-09-06T18:54:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '4lhMsR4INMw5LUlW+fjijg==', 'Content-Length': '3008833', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:vnxVSmpqWU4SKgYVVv2CZeWLgTU='} 2023-09-06T18:54:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.495 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.495 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.495 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.495 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.495 INFO:teuthology.orchestra.run.smithi099.stderr:ApVRZRTGuJLDF4R7yT+TnA== 2023-09-06T18:54:39.495 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.496 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.496 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/93 2023-09-06T18:54:39.496 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.496 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:T0yfDwfwBTB5XlDGzeKSZspocV0= 2023-09-06T18:54:39.497 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ApVRZRTGuJLDF4R7yT+TnA==', 'Content-Length': '3008841', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:T0yfDwfwBTB5XlDGzeKSZspocV0='} 2023-09-06T18:54:39.497 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.497 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.497 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.497 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.497 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.498 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.498 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.498 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.498 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.498 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1158.133929144 and callback=>; now=1128.133929144; delay=30.0 2023-09-06T18:54:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.501 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1158.145542482 and callback=>; now=1128.145542482; delay=30.0 2023-09-06T18:54:39.501 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.501 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.501 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.501 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.501 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.502 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.502 INFO:teuthology.orchestra.run.smithi099.stderr:FowqMMckz0VWUhuaNxe2nQ== 2023-09-06T18:54:39.502 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.502 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.502 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/60 2023-09-06T18:54:39.503 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.503 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1HpIPoejfKGUqWjAYirHX5A9szI= 2023-09-06T18:54:39.503 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'FowqMMckz0VWUhuaNxe2nQ==', 'Content-Length': '3011279', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1HpIPoejfKGUqWjAYirHX5A9szI='} 2023-09-06T18:54:39.503 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.503 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.504 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.504 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.504 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.504 INFO:teuthology.orchestra.run.smithi099.stderr:wH4O4G0cUa3RVZTZL8Bx8g== 2023-09-06T18:54:39.504 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.505 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.505 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/74 2023-09-06T18:54:39.505 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.505 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:oXpzU2EWQ1vcOuvfW97TE3ZNzJg= 2023-09-06T18:54:39.505 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'wH4O4G0cUa3RVZTZL8Bx8g==', 'Content-Length': '3009021', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:oXpzU2EWQ1vcOuvfW97TE3ZNzJg='} 2023-09-06T18:54:39.506 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.506 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.506 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.506 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.506 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.507 INFO:teuthology.orchestra.run.smithi099.stderr:Pynp8NrbQabMkK3jYh9YCQ== 2023-09-06T18:54:39.507 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.507 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.507 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/84 2023-09-06T18:54:39.507 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.508 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:DWb43RjPnXnEfLGJ6QXz3oIUtCk= 2023-09-06T18:54:39.508 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Pynp8NrbQabMkK3jYh9YCQ==', 'Content-Length': '3010540', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:DWb43RjPnXnEfLGJ6QXz3oIUtCk='} 2023-09-06T18:54:39.508 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.508 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.508 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.509 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.509 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.509 INFO:teuthology.orchestra.run.smithi099.stderr:bzcWCe4Zs+Wz9mmNWMtfMA== 2023-09-06T18:54:39.509 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.509 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.510 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/86 2023-09-06T18:54:39.510 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.510 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:jytVWq34l+SSBNBapWZMC66VzHY= 2023-09-06T18:54:39.510 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'bzcWCe4Zs+Wz9mmNWMtfMA==', 'Content-Length': '3011876', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:jytVWq34l+SSBNBapWZMC66VzHY='} 2023-09-06T18:54:39.510 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.511 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.511 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.511 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.511 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.511 INFO:teuthology.orchestra.run.smithi099.stderr:az8m3gLwA3rfuOuGoIUlIg== 2023-09-06T18:54:39.512 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.512 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.512 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/79 2023-09-06T18:54:39.512 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.512 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:g3Ny3rsaOzsW2WwvyD68FiQb99s= 2023-09-06T18:54:39.513 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'az8m3gLwA3rfuOuGoIUlIg==', 'Content-Length': '3009702', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:g3Ny3rsaOzsW2WwvyD68FiQb99s='} 2023-09-06T18:54:39.513 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.513 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.513 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.513 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.514 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.514 INFO:teuthology.orchestra.run.smithi099.stderr:13KPoJjsWZmb48Fg0K6R/g== 2023-09-06T18:54:39.514 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.514 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.514 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/75 2023-09-06T18:54:39.515 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.515 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:8wq87cdlwIhv6AMzvkexpnLie1o= 2023-09-06T18:54:39.515 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '13KPoJjsWZmb48Fg0K6R/g==', 'Content-Length': '3010936', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:8wq87cdlwIhv6AMzvkexpnLie1o='} 2023-09-06T18:54:39.515 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.515 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.515 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.516 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.516 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.516 INFO:teuthology.orchestra.run.smithi099.stderr:6Mq+4Y9CKsucB1afANko4g== 2023-09-06T18:54:39.516 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.516 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.517 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/90 2023-09-06T18:54:39.517 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.517 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1hcMZJN2zAi4C/ciWwcDP+nO928= 2023-09-06T18:54:39.517 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '6Mq+4Y9CKsucB1afANko4g==', 'Content-Length': '3013824', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1hcMZJN2zAi4C/ciWwcDP+nO928='} 2023-09-06T18:54:39.517 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.518 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.518 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.518 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.518 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.518 INFO:teuthology.orchestra.run.smithi099.stderr:ivwZ7Vdh/5AezzspgtVxkA== 2023-09-06T18:54:39.519 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.519 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.519 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/73 2023-09-06T18:54:39.519 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.519 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:rAgrUjCY/2UU1peWmmZ9tsZx824= 2023-09-06T18:54:39.520 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ivwZ7Vdh/5AezzspgtVxkA==', 'Content-Length': '3008097', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:rAgrUjCY/2UU1peWmmZ9tsZx824='} 2023-09-06T18:54:39.520 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.520 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.520 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.520 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:YDCq1EcJ1shxd4KL33U9ZA== 2023-09-06T18:54:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/85 2023-09-06T18:54:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.522 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:H//lau2lDgmRT3ZT1hifxD4FDCA= 2023-09-06T18:54:39.522 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'YDCq1EcJ1shxd4KL33U9ZA==', 'Content-Length': '3011189', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:H//lau2lDgmRT3ZT1hifxD4FDCA='} 2023-09-06T18:54:39.522 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.522 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.522 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.523 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.523 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.523 INFO:teuthology.orchestra.run.smithi099.stderr:ACjFwH5NGlKEdPWFDs0Uug== 2023-09-06T18:54:39.523 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.523 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.524 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/70 2023-09-06T18:54:39.524 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.524 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Gvl3cUJ2iRK2IcsrZZrixV76LeQ= 2023-09-06T18:54:39.524 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ACjFwH5NGlKEdPWFDs0Uug==', 'Content-Length': '3011260', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Gvl3cUJ2iRK2IcsrZZrixV76LeQ='} 2023-09-06T18:54:39.524 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.524 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.525 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.525 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.525 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.525 INFO:teuthology.orchestra.run.smithi099.stderr:dy8UZfYnVbeBuu0eEEEAdw== 2023-09-06T18:54:39.525 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.526 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.526 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/59 2023-09-06T18:54:39.526 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.526 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:FMb0hLbWXab8IMiyuTx2Sd1t5k4= 2023-09-06T18:54:39.526 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'dy8UZfYnVbeBuu0eEEEAdw==', 'Content-Length': '3013085', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:FMb0hLbWXab8IMiyuTx2Sd1t5k4='} 2023-09-06T18:54:39.527 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.527 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.527 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.527 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.527 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.528 INFO:teuthology.orchestra.run.smithi099.stderr:T5i766ZVJJ3eXXOUsc6lFA== 2023-09-06T18:54:39.528 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.528 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.528 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/63 2023-09-06T18:54:39.528 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.529 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:RYEZ13QtbXJqx9VanMv5lwSwvYo= 2023-09-06T18:54:39.529 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'T5i766ZVJJ3eXXOUsc6lFA==', 'Content-Length': '3010173', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:RYEZ13QtbXJqx9VanMv5lwSwvYo='} 2023-09-06T18:54:39.529 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.529 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.529 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.530 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.530 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.530 INFO:teuthology.orchestra.run.smithi099.stderr:1Wi/eLV38vm0BZtxAI2RNA== 2023-09-06T18:54:39.530 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.530 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.530 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/97 2023-09-06T18:54:39.531 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.531 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:dNHETCwYqsClR6tlXTHGA3vYsm8= 2023-09-06T18:54:39.531 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1Wi/eLV38vm0BZtxAI2RNA==', 'Content-Length': '3011110', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:dNHETCwYqsClR6tlXTHGA3vYsm8='} 2023-09-06T18:54:39.531 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.531 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.532 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.532 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.532 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.532 INFO:teuthology.orchestra.run.smithi099.stderr:d1WW15syDrKs3UdrMlkfWg== 2023-09-06T18:54:39.532 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.532 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.533 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/92 2023-09-06T18:54:39.533 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.533 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:4bsf40rLqAbfE0v5i1qQUo03ox4= 2023-09-06T18:54:39.533 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'd1WW15syDrKs3UdrMlkfWg==', 'Content-Length': '3010384', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:4bsf40rLqAbfE0v5i1qQUo03ox4='} 2023-09-06T18:54:39.533 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.534 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.534 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.534 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.534 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.534 INFO:teuthology.orchestra.run.smithi099.stderr:IlteDkzUvp19xkYkWXYCEw== 2023-09-06T18:54:39.535 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.535 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.535 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/95 2023-09-06T18:54:39.535 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.535 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:qU8kqZa6Kd3gi2mTN1yByHiSPhs= 2023-09-06T18:54:39.535 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'IlteDkzUvp19xkYkWXYCEw==', 'Content-Length': '3010755', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:qU8kqZa6Kd3gi2mTN1yByHiSPhs='} 2023-09-06T18:54:39.536 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.536 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.536 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.536 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.536 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.537 INFO:teuthology.orchestra.run.smithi099.stderr:TpxZegmN2A+sJ3sHtNR34g== 2023-09-06T18:54:39.537 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.537 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.537 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/81 2023-09-06T18:54:39.537 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.538 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6EPFOpoyygYmIMj+Rug1f0pP8YM= 2023-09-06T18:54:39.538 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TpxZegmN2A+sJ3sHtNR34g==', 'Content-Length': '3011797', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6EPFOpoyygYmIMj+Rug1f0pP8YM='} 2023-09-06T18:54:39.538 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.538 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.538 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.539 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.539 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.539 INFO:teuthology.orchestra.run.smithi099.stderr:TloLB9lPBB2/TtlB2Gy1wA== 2023-09-06T18:54:39.539 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.540 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.540 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/69 2023-09-06T18:54:39.540 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.540 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:6qcEOM3G1/zk11ieYYG76Z/6oiA= 2023-09-06T18:54:39.540 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'TloLB9lPBB2/TtlB2Gy1wA==', 'Content-Length': '3009018', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:6qcEOM3G1/zk11ieYYG76Z/6oiA='} 2023-09-06T18:54:39.540 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.541 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.541 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.541 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.541 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.541 INFO:teuthology.orchestra.run.smithi099.stderr:C6D+oxut6krwvUPn9E6aqQ== 2023-09-06T18:54:39.542 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.542 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.542 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/68 2023-09-06T18:54:39.542 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.542 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:CtTT22ewDAnC9YahPSmJI1/2Xt0= 2023-09-06T18:54:39.543 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C6D+oxut6krwvUPn9E6aqQ==', 'Content-Length': '3009821', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:CtTT22ewDAnC9YahPSmJI1/2Xt0='} 2023-09-06T18:54:39.543 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.543 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.543 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.543 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.544 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.544 INFO:teuthology.orchestra.run.smithi099.stderr:GJkR3zTdK3YwvIGvfcQsJA== 2023-09-06T18:54:39.544 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.544 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.544 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/94 2023-09-06T18:54:39.545 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.545 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:1TCHMs8abxmMC5byV+PH4dfiIn8= 2023-09-06T18:54:39.545 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GJkR3zTdK3YwvIGvfcQsJA==', 'Content-Length': '3011083', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:1TCHMs8abxmMC5byV+PH4dfiIn8='} 2023-09-06T18:54:39.545 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.545 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.546 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.546 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.546 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.546 INFO:teuthology.orchestra.run.smithi099.stderr:onB9dpXuID5Leh9ic6perg== 2023-09-06T18:54:39.546 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.547 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.547 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/71 2023-09-06T18:54:39.547 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.547 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Nu4fqXnsX73xXTrJxiVFnc1DqK8= 2023-09-06T18:54:39.547 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'onB9dpXuID5Leh9ic6perg==', 'Content-Length': '3010257', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Nu4fqXnsX73xXTrJxiVFnc1DqK8='} 2023-09-06T18:54:39.547 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.548 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.548 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.548 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.548 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.548 INFO:teuthology.orchestra.run.smithi099.stderr:H/7PVRaBhaiTgBGwsocA9g== 2023-09-06T18:54:39.549 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.549 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.549 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/66 2023-09-06T18:54:39.549 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.549 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:36FUK1BTY8M1HxewVhurZeS63eU= 2023-09-06T18:54:39.550 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'H/7PVRaBhaiTgBGwsocA9g==', 'Content-Length': '3010803', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:36FUK1BTY8M1HxewVhurZeS63eU='} 2023-09-06T18:54:39.550 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.550 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.550 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.550 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.551 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.551 INFO:teuthology.orchestra.run.smithi099.stderr:CExctBxSvJfXGKs/K/T9MQ== 2023-09-06T18:54:39.551 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.551 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.551 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/67 2023-09-06T18:54:39.551 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.552 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Kg+8r2tVv0ClHQIwVtGnpmy4f0A= 2023-09-06T18:54:39.552 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'CExctBxSvJfXGKs/K/T9MQ==', 'Content-Length': '3011407', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Kg+8r2tVv0ClHQIwVtGnpmy4f0A='} 2023-09-06T18:54:39.552 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.552 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.552 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.553 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.553 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.553 INFO:teuthology.orchestra.run.smithi099.stderr:hJWNhlOi09qLpvsl0uu7Cg== 2023-09-06T18:54:39.553 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.553 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.553 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/99 2023-09-06T18:54:39.554 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.554 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:Atvf3JbMyzqzFA+QlmjLbhjVYnk= 2023-09-06T18:54:39.554 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hJWNhlOi09qLpvsl0uu7Cg==', 'Content-Length': '3010645', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:Atvf3JbMyzqzFA+QlmjLbhjVYnk='} 2023-09-06T18:54:39.554 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.554 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.555 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.555 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.555 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.555 INFO:teuthology.orchestra.run.smithi099.stderr:LInDI/4s7vtCmmUdl2GZgQ== 2023-09-06T18:54:39.555 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.556 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.556 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/78 2023-09-06T18:54:39.556 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.556 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:wTYx6jGw8NbNYPV0mbzgPssE+PE= 2023-09-06T18:54:39.556 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'LInDI/4s7vtCmmUdl2GZgQ==', 'Content-Length': '3012430', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:wTYx6jGw8NbNYPV0mbzgPssE+PE='} 2023-09-06T18:54:39.557 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.557 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.557 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.557 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.557 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.557 INFO:teuthology.orchestra.run.smithi099.stderr:C/iQp/RnJcz00qfhYKVejg== 2023-09-06T18:54:39.558 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.558 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.558 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/83 2023-09-06T18:54:39.558 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.558 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VBQiPvpgkgY0eGav2GjAQBMnjks= 2023-09-06T18:54:39.559 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'C/iQp/RnJcz00qfhYKVejg==', 'Content-Length': '3008624', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VBQiPvpgkgY0eGav2GjAQBMnjks='} 2023-09-06T18:54:39.559 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.559 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.559 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.560 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.560 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.560 INFO:teuthology.orchestra.run.smithi099.stderr:OkRpNuHoekxqIV5d/AEmQw== 2023-09-06T18:54:39.560 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.560 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.561 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/96 2023-09-06T18:54:39.561 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.561 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:VNYADlT08pfUnJaj4kH1qEp0w0M= 2023-09-06T18:54:39.561 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'OkRpNuHoekxqIV5d/AEmQw==', 'Content-Length': '3010153', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:VNYADlT08pfUnJaj4kH1qEp0w0M='} 2023-09-06T18:54:39.561 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.561 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.562 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.562 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.562 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.562 INFO:teuthology.orchestra.run.smithi099.stderr:L+CnbtRBIsmcKjWa1bgqAQ== 2023-09-06T18:54:39.562 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.563 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:11 GMT 2023-09-06T18:54:39.563 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/62 2023-09-06T18:54:39.563 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.563 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:YvmJ7d/T+PZLDfTssCi26WRDB6U= 2023-09-06T18:54:39.563 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'L+CnbtRBIsmcKjWa1bgqAQ==', 'Content-Length': '3012829', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:YvmJ7d/T+PZLDfTssCi26WRDB6U='} 2023-09-06T18:54:39.563 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.564 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.564 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.564 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.564 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.564 INFO:teuthology.orchestra.run.smithi099.stderr:MILNOxKsGQkBzSCgDskNTA== 2023-09-06T18:54:39.565 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.565 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.565 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/80 2023-09-06T18:54:39.565 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.565 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:5sx0j0PgApPbvZFYmcsddlB3Dig= 2023-09-06T18:54:39.566 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'MILNOxKsGQkBzSCgDskNTA==', 'Content-Length': '3009370', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:5sx0j0PgApPbvZFYmcsddlB3Dig='} 2023-09-06T18:54:39.566 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.566 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.566 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.566 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.567 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.567 INFO:teuthology.orchestra.run.smithi099.stderr:vUxwqUKsZaB1t8UY3dtHOg== 2023-09-06T18:54:39.567 INFO:teuthology.orchestra.run.smithi099.stderr:application/octet-stream 2023-09-06T18:54:39.567 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:41:12 GMT 2023-09-06T18:54:39.567 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/87 2023-09-06T18:54:39.568 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.568 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:EBRZ6AH9z0Bo+QZREF5CA5CZ9XA= 2023-09-06T18:54:39.568 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'vUxwqUKsZaB1t8UY3dtHOg==', 'Content-Length': '3009715', 'Expect': '100-Continue', 'Date': 'Wed, 06 Sep 2023 18:41:12 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:EBRZ6AH9z0Bo+QZREF5CA5CZ9XA='} 2023-09-06T18:54:39.568 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.568 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.569 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.569 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.569 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.569 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.569 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.570 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.570 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.570 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1188.13476297 and callback=>; now=1158.13476297; delay=30.0 2023-09-06T18:54:39.570 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.570 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.571 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.571 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.571 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.571 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.571 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.572 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.572 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.572 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.572 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1188.146511382 and callback=>; now=1158.146511382; delay=30.0 2023-09-06T18:54:39.572 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.573 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.573 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.573 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 2 heartbeat frames, sent 2, idle intervals 0 2023-09-06T18:54:39.573 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1228.137804521 and callback=>; now=1163.137804521; delay=65 2023-09-06T18:54:39.573 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 2 heartbeat frames, sent 2, idle intervals 0 2023-09-06T18:54:39.574 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1228.149237745 and callback=>; now=1163.149237745; delay=65 2023-09-06T18:54:39.574 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.574 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.574 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.574 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.575 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.575 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.575 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.575 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1218.135696911 and callback=>; now=1188.135696911; delay=30.0 2023-09-06T18:54:39.575 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.576 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.576 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.576 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.576 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.576 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.577 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.577 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.577 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.577 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.577 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1218.147514739 and callback=>; now=1188.147514739; delay=30.0 2023-09-06T18:54:39.578 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.578 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.578 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.578 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-2/ 2023-09-06T18:54:39.578 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-2/ 2023-09-06T18:54:39.581 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: GET 2023-09-06T18:54:39.581 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-2/ 2023-09-06T18:54:39.581 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:39.581 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {} 2023-09-06T18:54:39.581 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:39.582 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:39.582 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:39.582 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.582 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.582 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.583 INFO:teuthology.orchestra.run.smithi099.stderr:GET 2023-09-06T18:54:39.583 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.583 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.583 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:43:07 GMT 2023-09-06T18:54:39.583 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/ 2023-09-06T18:54:39.584 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.584 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:FOP1MbGdaIJhW7C0joE+FvIBh24= 2023-09-06T18:54:39.584 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:43:07 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:FOP1MbGdaIJhW7C0joE+FvIBh24=', 'Content-Length': '0'} 2023-09-06T18:54:39.584 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.584 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.585 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.585 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.585 INFO:teuthology.orchestra.run.smithi099.stderr:GET 2023-09-06T18:54:39.585 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.585 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.585 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:43:07 GMT 2023-09-06T18:54:39.586 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/ 2023-09-06T18:54:39.586 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.586 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:FOP1MbGdaIJhW7C0joE+FvIBh24= 2023-09-06T18:54:39.586 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:43:07 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:FOP1MbGdaIJhW7C0joE+FvIBh24=', 'Content-Length': '0'} 2023-09-06T18:54:39.586 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.587 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.587 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.587 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.587 INFO:teuthology.orchestra.run.smithi099.stderr:GET 2023-09-06T18:54:39.587 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.588 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.588 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:43:07 GMT 2023-09-06T18:54:39.588 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/ 2023-09-06T18:54:39.588 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.588 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:FOP1MbGdaIJhW7C0joE+FvIBh24= 2023-09-06T18:54:39.589 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:43:07 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:FOP1MbGdaIJhW7C0joE+FvIBh24=', 'Content-Length': '0'} 2023-09-06T18:54:39.589 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.589 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.589 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.589 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.589 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.590 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.590 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.590 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.590 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.590 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1248.13673103 and callback=>; now=1218.13673103; delay=30.0 2023-09-06T18:54:39.591 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.591 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.591 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.591 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.592 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.592 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.592 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.592 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.592 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.593 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.593 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1248.148413371 and callback=>; now=1218.148413371; delay=30.0 2023-09-06T18:54:39.593 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.593 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.593 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.593 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.594 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.594 INFO:teuthology.orchestra.run.smithi099.stderr:GET 2023-09-06T18:54:39.594 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.594 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.594 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:43:07 GMT 2023-09-06T18:54:39.595 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/ 2023-09-06T18:54:39.595 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.595 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:FOP1MbGdaIJhW7C0joE+FvIBh24= 2023-09-06T18:54:39.595 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:43:07 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:FOP1MbGdaIJhW7C0joE+FvIBh24=', 'Content-Length': '0'} 2023-09-06T18:54:39.595 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.596 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.596 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.596 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.596 INFO:teuthology.orchestra.run.smithi099.stderr:GET 2023-09-06T18:54:39.596 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.596 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.597 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:43:07 GMT 2023-09-06T18:54:39.597 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/ 2023-09-06T18:54:39.597 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.597 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:FOP1MbGdaIJhW7C0joE+FvIBh24= 2023-09-06T18:54:39.598 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:43:07 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:FOP1MbGdaIJhW7C0joE+FvIBh24=', 'Content-Length': '0'} 2023-09-06T18:54:39.598 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.598 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.598 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 4 heartbeat frames, sent 4, idle intervals 0 2023-09-06T18:54:39.598 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1293.143541248 and callback=>; now=1228.143541248; delay=65 2023-09-06T18:54:39.598 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 4 heartbeat frames, sent 4, idle intervals 0 2023-09-06T18:54:39.599 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1293.155078577 and callback=>; now=1228.155078577; delay=65 2023-09-06T18:54:39.599 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.599 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.599 INFO:teuthology.orchestra.run.smithi099.stderr:GET 2023-09-06T18:54:39.599 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.600 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.600 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:43:07 GMT 2023-09-06T18:54:39.600 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/ 2023-09-06T18:54:39.600 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.600 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:FOP1MbGdaIJhW7C0joE+FvIBh24= 2023-09-06T18:54:39.601 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:43:07 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:FOP1MbGdaIJhW7C0joE+FvIBh24=', 'Content-Length': '0'} 2023-09-06T18:54:39.601 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.601 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.601 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.601 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.602 INFO:teuthology.orchestra.run.smithi099.stderr:GET 2023-09-06T18:54:39.602 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.602 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.602 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:43:07 GMT 2023-09-06T18:54:39.602 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-2/ 2023-09-06T18:54:39.603 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.603 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:FOP1MbGdaIJhW7C0joE+FvIBh24= 2023-09-06T18:54:39.603 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:43:07 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:FOP1MbGdaIJhW7C0joE+FvIBh24=', 'Content-Length': '0'} 2023-09-06T18:54:39.603 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.603 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.603 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.604 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.604 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.604 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.604 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.604 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.605 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.605 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1278.137692006 and callback=>; now=1248.137692006; delay=30.0 2023-09-06T18:54:39.605 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.605 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.605 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.606 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.606 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.606 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.606 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.607 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.607 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.607 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.607 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1278.149614695 and callback=>; now=1248.149614695; delay=30.0 2023-09-06T18:54:39.607 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.607 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.609 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.609 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.609 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.609 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1308.138665433 and callback=>; now=1278.138665433; delay=30.0 2023-09-06T18:54:39.609 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.610 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.610 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.610 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.610 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.610 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.611 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.611 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.611 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.611 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.611 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1308.15037015 and callback=>; now=1278.15037015; delay=30.0 2023-09-06T18:54:39.611 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.612 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.612 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.612 INFO:teuthology.orchestra.run.smithi099.stderr:--------------------- >> end captured logging << --------------------- 2023-09-06T18:54:39.612 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.612 INFO:teuthology.orchestra.run.smithi099.stderr:====================================================================== 2023-09-06T18:54:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:ERROR: bucket_notification.test_bn.test_ps_s3_creation_triggers_on_master 2023-09-06T18:54:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:---------------------------------------------------------------------- 2023-09-06T18:54:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.613 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest 2023-09-06T18:54:39.613 INFO:teuthology.orchestra.run.smithi099.stderr: self.test(*self.arg) 2023-09-06T18:54:39.614 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 2137, in test_ps_s3_creation_triggers_on_master 2023-09-06T18:54:39.614 INFO:teuthology.orchestra.run.smithi099.stderr: ps_s3_creation_triggers_on_master(external_endpoint_address="amqp://localhost:5672") 2023-09-06T18:54:39.614 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 2061, in ps_s3_creation_triggers_on_master 2023-09-06T18:54:39.614 INFO:teuthology.orchestra.run.smithi099.stderr: bucket = conn.create_bucket(bucket_name) 2023-09-06T18:54:39.614 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 619, in create_bucket 2023-09-06T18:54:39.615 INFO:teuthology.orchestra.run.smithi099.stderr: data=data) 2023-09-06T18:54:39.615 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 671, in make_request 2023-09-06T18:54:39.615 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler 2023-09-06T18:54:39.615 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1071, in make_request 2023-09-06T18:54:39.615 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler) 2023-09-06T18:54:39.616 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1030, in _mexe 2023-09-06T18:54:39.616 INFO:teuthology.orchestra.run.smithi099.stderr: raise ex 2023-09-06T18:54:39.616 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 943, in _mexe 2023-09-06T18:54:39.616 INFO:teuthology.orchestra.run.smithi099.stderr: request.body, request.headers) 2023-09-06T18:54:39.616 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.616 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.617 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.617 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.617 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.617 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.617 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1044, in _send_output 2023-09-06T18:54:39.618 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.618 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.618 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.618 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 954, in connect 2023-09-06T18:54:39.618 INFO:teuthology.orchestra.run.smithi099.stderr: (self.host,self.port), self.timeout, self.source_address) 2023-09-06T18:54:39.619 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 724, in create_connection 2023-09-06T18:54:39.619 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.619 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 713, in create_connection 2023-09-06T18:54:39.619 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.619 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:-------------------- >> begin captured logging << -------------------- 2023-09-06T18:54:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using access key provided by client. 2023-09-06T18:54:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using secret key provided by client. 2023-09-06T18:54:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-3/ 2023-09-06T18:54:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-3/ 2023-09-06T18:54:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-3/ 2023-09-06T18:54:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {} 2023-09-06T18:54:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:39.622 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:39.622 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.622 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.622 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.622 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.623 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.623 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.623 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:44:13 GMT 2023-09-06T18:54:39.623 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-3/ 2023-09-06T18:54:39.623 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.624 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:jd0eq3Os8RiGnt3MX+aA3wgEXMY= 2023-09-06T18:54:39.624 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:44:13 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:jd0eq3Os8RiGnt3MX+aA3wgEXMY=', 'Content-Length': '0'} 2023-09-06T18:54:39.624 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.624 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.624 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.624 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.625 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.625 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.625 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.625 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:44:13 GMT 2023-09-06T18:54:39.625 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-3/ 2023-09-06T18:54:39.626 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.626 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:jd0eq3Os8RiGnt3MX+aA3wgEXMY= 2023-09-06T18:54:39.626 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:44:13 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:jd0eq3Os8RiGnt3MX+aA3wgEXMY=', 'Content-Length': '0'} 2023-09-06T18:54:39.626 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.626 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.627 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.627 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.627 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.627 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.627 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.628 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:44:13 GMT 2023-09-06T18:54:39.628 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-3/ 2023-09-06T18:54:39.628 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.628 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:jd0eq3Os8RiGnt3MX+aA3wgEXMY= 2023-09-06T18:54:39.628 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:44:13 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:jd0eq3Os8RiGnt3MX+aA3wgEXMY=', 'Content-Length': '0'} 2023-09-06T18:54:39.629 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.629 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.629 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.629 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.629 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.629 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.630 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.630 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:44:13 GMT 2023-09-06T18:54:39.630 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-3/ 2023-09-06T18:54:39.630 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.630 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:jd0eq3Os8RiGnt3MX+aA3wgEXMY= 2023-09-06T18:54:39.631 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:44:13 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:jd0eq3Os8RiGnt3MX+aA3wgEXMY=', 'Content-Length': '0'} 2023-09-06T18:54:39.631 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.631 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.631 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.631 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.632 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.632 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.632 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.632 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:44:13 GMT 2023-09-06T18:54:39.632 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-3/ 2023-09-06T18:54:39.632 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.633 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:jd0eq3Os8RiGnt3MX+aA3wgEXMY= 2023-09-06T18:54:39.633 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:44:13 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:jd0eq3Os8RiGnt3MX+aA3wgEXMY=', 'Content-Length': '0'} 2023-09-06T18:54:39.633 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.633 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.633 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 6 heartbeat frames, sent 6, idle intervals 0 2023-09-06T18:54:39.634 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1358.149646876 and callback=>; now=1293.149646876; delay=65 2023-09-06T18:54:39.634 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 6 heartbeat frames, sent 6, idle intervals 0 2023-09-06T18:54:39.634 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1358.16035781 and callback=>; now=1293.16035781; delay=65 2023-09-06T18:54:39.634 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.634 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.635 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.635 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.635 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.635 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:44:13 GMT 2023-09-06T18:54:39.635 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-3/ 2023-09-06T18:54:39.636 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.636 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:jd0eq3Os8RiGnt3MX+aA3wgEXMY= 2023-09-06T18:54:39.636 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:44:13 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:jd0eq3Os8RiGnt3MX+aA3wgEXMY=', 'Content-Length': '0'} 2023-09-06T18:54:39.636 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.636 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.636 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.637 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.637 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.637 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.637 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.637 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.638 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.638 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1338.139732205 and callback=>; now=1308.139732205; delay=30.0 2023-09-06T18:54:39.638 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.638 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.638 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.640 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.640 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.640 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1338.151381023 and callback=>; now=1308.151381023; delay=30.0 2023-09-06T18:54:39.640 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.640 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.641 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.641 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.641 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.641 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.641 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.641 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:44:13 GMT 2023-09-06T18:54:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-3/ 2023-09-06T18:54:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:jd0eq3Os8RiGnt3MX+aA3wgEXMY= 2023-09-06T18:54:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:44:13 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:jd0eq3Os8RiGnt3MX+aA3wgEXMY=', 'Content-Length': '0'} 2023-09-06T18:54:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1368.140638296 and callback=>; now=1338.140638296; delay=30.0 2023-09-06T18:54:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1368.152598917 and callback=>; now=1338.152598917; delay=30.0 2023-09-06T18:54:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 8 heartbeat frames, sent 8, idle intervals 0 2023-09-06T18:54:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1423.155667717 and callback=>; now=1358.155667717; delay=65 2023-09-06T18:54:39.648 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 8 heartbeat frames, sent 8, idle intervals 0 2023-09-06T18:54:39.648 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1423.16630011 and callback=>; now=1358.16630011; delay=65 2023-09-06T18:54:39.648 INFO:teuthology.orchestra.run.smithi099.stderr:--------------------- >> end captured logging << --------------------- 2023-09-06T18:54:39.648 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.648 INFO:teuthology.orchestra.run.smithi099.stderr:====================================================================== 2023-09-06T18:54:39.649 INFO:teuthology.orchestra.run.smithi099.stderr:ERROR: test that uploads object using HTTP POST 2023-09-06T18:54:39.649 INFO:teuthology.orchestra.run.smithi099.stderr:---------------------------------------------------------------------- 2023-09-06T18:54:39.649 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.649 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest 2023-09-06T18:54:39.649 INFO:teuthology.orchestra.run.smithi099.stderr: self.test(*self.arg) 2023-09-06T18:54:39.650 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 2310, in test_http_post_object_upload 2023-09-06T18:54:39.650 INFO:teuthology.orchestra.run.smithi099.stderr: bucket = conn1.create_bucket(ACL='public-read-write', Bucket=bucket_name) 2023-09-06T18:54:39.650 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/client.py", line 508, in _api_call 2023-09-06T18:54:39.650 INFO:teuthology.orchestra.run.smithi099.stderr: return self._make_api_call(operation_name, kwargs) 2023-09-06T18:54:39.650 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/client.py", line 895, in _make_api_call 2023-09-06T18:54:39.650 INFO:teuthology.orchestra.run.smithi099.stderr: operation_model, request_dict, request_context 2023-09-06T18:54:39.651 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/client.py", line 917, in _make_request 2023-09-06T18:54:39.651 INFO:teuthology.orchestra.run.smithi099.stderr: return self._endpoint.make_request(operation_model, request_dict) 2023-09-06T18:54:39.651 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 116, in make_request 2023-09-06T18:54:39.651 INFO:teuthology.orchestra.run.smithi099.stderr: return self._send_request(request_dict, operation_model) 2023-09-06T18:54:39.651 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 204, in _send_request 2023-09-06T18:54:39.652 INFO:teuthology.orchestra.run.smithi099.stderr: exception, 2023-09-06T18:54:39.652 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 358, in _needs_retry 2023-09-06T18:54:39.652 INFO:teuthology.orchestra.run.smithi099.stderr: request_dict=request_dict, 2023-09-06T18:54:39.652 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/hooks.py", line 412, in emit 2023-09-06T18:54:39.652 INFO:teuthology.orchestra.run.smithi099.stderr: return self._emitter.emit(aliased_event_name, **kwargs) 2023-09-06T18:54:39.653 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/hooks.py", line 256, in emit 2023-09-06T18:54:39.653 INFO:teuthology.orchestra.run.smithi099.stderr: return self._emit(event_name, kwargs) 2023-09-06T18:54:39.653 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/hooks.py", line 239, in _emit 2023-09-06T18:54:39.653 INFO:teuthology.orchestra.run.smithi099.stderr: response = handler(**kwargs) 2023-09-06T18:54:39.653 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 207, in __call__ 2023-09-06T18:54:39.654 INFO:teuthology.orchestra.run.smithi099.stderr: if self._checker(**checker_kwargs): 2023-09-06T18:54:39.654 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 285, in __call__ 2023-09-06T18:54:39.654 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, response, caught_exception 2023-09-06T18:54:39.654 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 320, in _should_retry 2023-09-06T18:54:39.654 INFO:teuthology.orchestra.run.smithi099.stderr: return self._checker(attempt_number, response, caught_exception) 2023-09-06T18:54:39.655 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 364, in __call__ 2023-09-06T18:54:39.655 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, response, caught_exception 2023-09-06T18:54:39.655 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 248, in __call__ 2023-09-06T18:54:39.655 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, caught_exception 2023-09-06T18:54:39.655 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 416, in _check_caught_exception 2023-09-06T18:54:39.656 INFO:teuthology.orchestra.run.smithi099.stderr: raise caught_exception 2023-09-06T18:54:39.656 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 278, in _do_get_response 2023-09-06T18:54:39.656 INFO:teuthology.orchestra.run.smithi099.stderr: http_response = self._send(request) 2023-09-06T18:54:39.656 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 374, in _send 2023-09-06T18:54:39.656 INFO:teuthology.orchestra.run.smithi099.stderr: return self.http_session.send(request) 2023-09-06T18:54:39.657 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 472, in send 2023-09-06T18:54:39.657 INFO:teuthology.orchestra.run.smithi099.stderr: raise EndpointConnectionError(endpoint_url=request.url, error=e) 2023-09-06T18:54:39.657 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://smithi099.front.sepia.ceph.com:80/cjhpse-4" 2023-09-06T18:54:39.657 INFO:teuthology.orchestra.run.smithi099.stderr:-------------------- >> begin captured logging << -------------------- 2023-09-06T18:54:39.657 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using access key provided by client. 2023-09-06T18:54:39.657 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using secret key provided by client. 2023-09-06T18:54:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-service-name: calling handler 2023-09-06T18:54:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler 2023-09-06T18:54:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler ._handler at 0x7f5e9314f378> 2023-09-06T18:54:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler 2023-09-06T18:54:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Setting s3 timeout as (60, 60) 2023-09-06T18:54:39.659 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.client: DEBUG: Registering retry handlers for service: s3 2023-09-06T18:54:39.659 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.PutObject: calling handler > 2023-09-06T18:54:39.659 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.PutObject: calling handler > 2023-09-06T18:54:39.659 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-sign.s3.PutObject: calling handler > 2023-09-06T18:54:39.659 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.utils: DEBUG: Using S3 path style addressing. 2023-09-06T18:54:39.660 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-parameter-build.s3.CreateBucket: calling handler 2023-09-06T18:54:39.660 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-parameter-build.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.660 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-parameter-build.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.660 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-parameter-build.s3.CreateBucket: calling handler 2023-09-06T18:54:39.660 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-call.s3.CreateBucket: calling handler 2023-09-06T18:54:39.661 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-call.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.661 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-call.s3.CreateBucket: calling handler 2023-09-06T18:54:39.661 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-call.s3.CreateBucket: calling handler 2023-09-06T18:54:39.661 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Making request for OperationModel(name=CreateBucket) with params: {'url_path': '/cjhpse-4', 'query_string': {}, 'method': 'PUT', 'headers': {'x-amz-acl': 'public-read-write', 'User-Agent': 'Boto3/1.23.10 Python/3.6.8 Linux/4.18.0-513.el8.x86_64 Botocore/1.26.10'}, 'body': b'', 'url': 'http://smithi099.front.sepia.ceph.com:80/cjhpse-4', 'context': {'client_region': 'us-east-1', 'client_config': , 'has_streaming_input': False, 'auth_type': None, 'signing': {'bucket': 'cjhpse-4'}}} 2023-09-06T18:54:39.661 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.661 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.662 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.662 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler 2023-09-06T18:54:39.662 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-sign.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.662 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Calculating signature using v4 auth. 2023-09-06T18:54:39.662 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: CanonicalRequest: 2023-09-06T18:54:39.663 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.663 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-4 2023-09-06T18:54:39.663 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.663 INFO:teuthology.orchestra.run.smithi099.stderr:host:smithi099.front.sepia.ceph.com 2023-09-06T18:54:39.663 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-acl:public-read-write 2023-09-06T18:54:39.664 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2023-09-06T18:54:39.664 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-date:20230906T184539Z 2023-09-06T18:54:39.664 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.664 INFO:teuthology.orchestra.run.smithi099.stderr:host;x-amz-acl;x-amz-content-sha256;x-amz-date 2023-09-06T18:54:39.664 INFO:teuthology.orchestra.run.smithi099.stderr:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2023-09-06T18:54:39.664 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: StringToSign: 2023-09-06T18:54:39.665 INFO:teuthology.orchestra.run.smithi099.stderr:AWS4-HMAC-SHA256 2023-09-06T18:54:39.665 INFO:teuthology.orchestra.run.smithi099.stderr:20230906T184539Z 2023-09-06T18:54:39.665 INFO:teuthology.orchestra.run.smithi099.stderr:20230906/us-east-1/s3/aws4_request 2023-09-06T18:54:39.665 INFO:teuthology.orchestra.run.smithi099.stderr:7afe4cae52f1516d9b63374d6fd1d484da3058f1e340c1ee3c18a2ef409fb321 2023-09-06T18:54:39.665 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Signature: 2023-09-06T18:54:39.665 INFO:teuthology.orchestra.run.smithi099.stderr:ca928b4f0f4014fd2be619cab70e81ea529cf7f16b3e864cc6a292c1d9de18c2 2023-09-06T18:54:39.666 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.s3.CreateBucket: calling handler 2023-09-06T18:54:39.666 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Sending http request: 2023-09-06T18:54:39.666 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): smithi099.front.sepia.ceph.com:80 2023-09-06T18:54:39.666 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Exception received when sending HTTP request. 2023-09-06T18:54:39.667 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.667 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 175, in _new_conn 2023-09-06T18:54:39.667 INFO:teuthology.orchestra.run.smithi099.stderr: (self._dns_host, self.port), self.timeout, **extra_kw 2023-09-06T18:54:39.667 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 95, in create_connection 2023-09-06T18:54:39.667 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.667 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 85, in create_connection 2023-09-06T18:54:39.668 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.668 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.668 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.668 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.668 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.669 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.669 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 452, in send 2023-09-06T18:54:39.669 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=self._chunked(request.headers), 2023-09-06T18:54:39.669 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 799, in urlopen 2023-09-06T18:54:39.669 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] 2023-09-06T18:54:39.670 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/retry.py", line 525, in increment 2023-09-06T18:54:39.670 INFO:teuthology.orchestra.run.smithi099.stderr: raise six.reraise(type(error), error, _stacktrace) 2023-09-06T18:54:39.670 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/packages/six.py", line 770, in reraise 2023-09-06T18:54:39.670 INFO:teuthology.orchestra.run.smithi099.stderr: raise value 2023-09-06T18:54:39.670 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 721, in urlopen 2023-09-06T18:54:39.671 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=chunked, 2023-09-06T18:54:39.671 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 415, in _make_request 2023-09-06T18:54:39.671 INFO:teuthology.orchestra.run.smithi099.stderr: conn.request(method, url, **httplib_request_kw) 2023-09-06T18:54:39.671 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 244, in request 2023-09-06T18:54:39.671 INFO:teuthology.orchestra.run.smithi099.stderr: super(HTTPConnection, self).request(method, url, body=body, headers=headers) 2023-09-06T18:54:39.672 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.672 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.672 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 95, in _send_request 2023-09-06T18:54:39.672 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, body, headers, *args, **kwargs 2023-09-06T18:54:39.672 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.673 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.673 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.673 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.673 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 123, in _send_output 2023-09-06T18:54:39.673 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.673 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 218, in send 2023-09-06T18:54:39.674 INFO:teuthology.orchestra.run.smithi099.stderr: return super().send(str) 2023-09-06T18:54:39.674 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.674 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.674 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 205, in connect 2023-09-06T18:54:39.674 INFO:teuthology.orchestra.run.smithi099.stderr: conn = self._new_conn() 2023-09-06T18:54:39.675 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 187, in _new_conn 2023-09-06T18:54:39.675 INFO:teuthology.orchestra.run.smithi099.stderr: self, "Failed to establish a new connection: %s" % e 2023-09-06T18:54:39.675 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused 2023-09-06T18:54:39.675 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.675 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.676 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.676 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.676 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 278, in _do_get_response 2023-09-06T18:54:39.676 INFO:teuthology.orchestra.run.smithi099.stderr: http_response = self._send(request) 2023-09-06T18:54:39.676 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 374, in _send 2023-09-06T18:54:39.677 INFO:teuthology.orchestra.run.smithi099.stderr: return self.http_session.send(request) 2023-09-06T18:54:39.677 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 472, in send 2023-09-06T18:54:39.677 INFO:teuthology.orchestra.run.smithi099.stderr: raise EndpointConnectionError(endpoint_url=request.url, error=e) 2023-09-06T18:54:39.677 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://smithi099.front.sepia.ceph.com:80/cjhpse-4" 2023-09-06T18:54:39.677 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event needs-retry.s3.CreateBucket: calling handler 2023-09-06T18:54:39.677 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.retryhandler: DEBUG: retry needed, retryable exception caught: Could not connect to the endpoint URL: "http://smithi099.front.sepia.ceph.com:80/cjhpse-4" 2023-09-06T18:54:39.678 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.678 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 175, in _new_conn 2023-09-06T18:54:39.678 INFO:teuthology.orchestra.run.smithi099.stderr: (self._dns_host, self.port), self.timeout, **extra_kw 2023-09-06T18:54:39.678 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 95, in create_connection 2023-09-06T18:54:39.678 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.679 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 85, in create_connection 2023-09-06T18:54:39.679 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.679 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.679 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.679 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.680 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.680 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.680 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 452, in send 2023-09-06T18:54:39.680 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=self._chunked(request.headers), 2023-09-06T18:54:39.680 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 799, in urlopen 2023-09-06T18:54:39.681 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] 2023-09-06T18:54:39.681 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/retry.py", line 525, in increment 2023-09-06T18:54:39.681 INFO:teuthology.orchestra.run.smithi099.stderr: raise six.reraise(type(error), error, _stacktrace) 2023-09-06T18:54:39.681 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/packages/six.py", line 770, in reraise 2023-09-06T18:54:39.681 INFO:teuthology.orchestra.run.smithi099.stderr: raise value 2023-09-06T18:54:39.681 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 721, in urlopen 2023-09-06T18:54:39.682 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=chunked, 2023-09-06T18:54:39.682 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 415, in _make_request 2023-09-06T18:54:39.682 INFO:teuthology.orchestra.run.smithi099.stderr: conn.request(method, url, **httplib_request_kw) 2023-09-06T18:54:39.682 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 244, in request 2023-09-06T18:54:39.683 INFO:teuthology.orchestra.run.smithi099.stderr: super(HTTPConnection, self).request(method, url, body=body, headers=headers) 2023-09-06T18:54:39.683 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.683 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.683 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 95, in _send_request 2023-09-06T18:54:39.683 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, body, headers, *args, **kwargs 2023-09-06T18:54:39.684 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.684 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.684 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.684 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.684 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 123, in _send_output 2023-09-06T18:54:39.685 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.685 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 218, in send 2023-09-06T18:54:39.685 INFO:teuthology.orchestra.run.smithi099.stderr: return super().send(str) 2023-09-06T18:54:39.685 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.685 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.686 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 205, in connect 2023-09-06T18:54:39.686 INFO:teuthology.orchestra.run.smithi099.stderr: conn = self._new_conn() 2023-09-06T18:54:39.686 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 187, in _new_conn 2023-09-06T18:54:39.686 INFO:teuthology.orchestra.run.smithi099.stderr: self, "Failed to establish a new connection: %s" % e 2023-09-06T18:54:39.686 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused 2023-09-06T18:54:39.686 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.687 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.687 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.687 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.687 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 308, in _should_retry 2023-09-06T18:54:39.687 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, response, caught_exception 2023-09-06T18:54:39.688 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 364, in __call__ 2023-09-06T18:54:39.688 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, response, caught_exception 2023-09-06T18:54:39.688 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 248, in __call__ 2023-09-06T18:54:39.688 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, caught_exception 2023-09-06T18:54:39.688 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 416, in _check_caught_exception 2023-09-06T18:54:39.689 INFO:teuthology.orchestra.run.smithi099.stderr: raise caught_exception 2023-09-06T18:54:39.689 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 278, in _do_get_response 2023-09-06T18:54:39.689 INFO:teuthology.orchestra.run.smithi099.stderr: http_response = self._send(request) 2023-09-06T18:54:39.689 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 374, in _send 2023-09-06T18:54:39.689 INFO:teuthology.orchestra.run.smithi099.stderr: return self.http_session.send(request) 2023-09-06T18:54:39.690 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 472, in send 2023-09-06T18:54:39.690 INFO:teuthology.orchestra.run.smithi099.stderr: raise EndpointConnectionError(endpoint_url=request.url, error=e) 2023-09-06T18:54:39.690 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://smithi099.front.sepia.ceph.com:80/cjhpse-4" 2023-09-06T18:54:39.690 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.retryhandler: DEBUG: Retry needed, action of: 0.206388257726285 2023-09-06T18:54:39.690 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event needs-retry.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.691 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Response received to retry, sleeping for 0.206388257726285 seconds 2023-09-06T18:54:39.691 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.691 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.691 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.692 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler 2023-09-06T18:54:39.692 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-sign.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.692 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Calculating signature using v4 auth. 2023-09-06T18:54:39.692 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: CanonicalRequest: 2023-09-06T18:54:39.692 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.693 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-4 2023-09-06T18:54:39.693 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.693 INFO:teuthology.orchestra.run.smithi099.stderr:host:smithi099.front.sepia.ceph.com 2023-09-06T18:54:39.693 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-acl:public-read-write 2023-09-06T18:54:39.693 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2023-09-06T18:54:39.693 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-date:20230906T184539Z 2023-09-06T18:54:39.694 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.694 INFO:teuthology.orchestra.run.smithi099.stderr:host;x-amz-acl;x-amz-content-sha256;x-amz-date 2023-09-06T18:54:39.694 INFO:teuthology.orchestra.run.smithi099.stderr:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2023-09-06T18:54:39.694 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: StringToSign: 2023-09-06T18:54:39.694 INFO:teuthology.orchestra.run.smithi099.stderr:AWS4-HMAC-SHA256 2023-09-06T18:54:39.695 INFO:teuthology.orchestra.run.smithi099.stderr:20230906T184539Z 2023-09-06T18:54:39.695 INFO:teuthology.orchestra.run.smithi099.stderr:20230906/us-east-1/s3/aws4_request 2023-09-06T18:54:39.695 INFO:teuthology.orchestra.run.smithi099.stderr:7afe4cae52f1516d9b63374d6fd1d484da3058f1e340c1ee3c18a2ef409fb321 2023-09-06T18:54:39.695 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Signature: 2023-09-06T18:54:39.695 INFO:teuthology.orchestra.run.smithi099.stderr:ca928b4f0f4014fd2be619cab70e81ea529cf7f16b3e864cc6a292c1d9de18c2 2023-09-06T18:54:39.695 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.s3.CreateBucket: calling handler 2023-09-06T18:54:39.696 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Sending http request: 2023-09-06T18:54:39.696 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.connectionpool: DEBUG: Starting new HTTP connection (2): smithi099.front.sepia.ceph.com:80 2023-09-06T18:54:39.696 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Exception received when sending HTTP request. 2023-09-06T18:54:39.696 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.696 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 175, in _new_conn 2023-09-06T18:54:39.697 INFO:teuthology.orchestra.run.smithi099.stderr: (self._dns_host, self.port), self.timeout, **extra_kw 2023-09-06T18:54:39.697 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 95, in create_connection 2023-09-06T18:54:39.697 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.697 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 85, in create_connection 2023-09-06T18:54:39.697 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.698 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.698 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.698 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.698 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.698 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.699 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 452, in send 2023-09-06T18:54:39.699 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=self._chunked(request.headers), 2023-09-06T18:54:39.699 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 799, in urlopen 2023-09-06T18:54:39.699 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] 2023-09-06T18:54:39.699 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/retry.py", line 525, in increment 2023-09-06T18:54:39.700 INFO:teuthology.orchestra.run.smithi099.stderr: raise six.reraise(type(error), error, _stacktrace) 2023-09-06T18:54:39.700 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/packages/six.py", line 770, in reraise 2023-09-06T18:54:39.700 INFO:teuthology.orchestra.run.smithi099.stderr: raise value 2023-09-06T18:54:39.700 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 721, in urlopen 2023-09-06T18:54:39.700 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=chunked, 2023-09-06T18:54:39.701 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 415, in _make_request 2023-09-06T18:54:39.701 INFO:teuthology.orchestra.run.smithi099.stderr: conn.request(method, url, **httplib_request_kw) 2023-09-06T18:54:39.701 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 244, in request 2023-09-06T18:54:39.701 INFO:teuthology.orchestra.run.smithi099.stderr: super(HTTPConnection, self).request(method, url, body=body, headers=headers) 2023-09-06T18:54:39.701 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.702 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.702 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 95, in _send_request 2023-09-06T18:54:39.702 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, body, headers, *args, **kwargs 2023-09-06T18:54:39.702 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.702 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.703 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.703 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.703 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 123, in _send_output 2023-09-06T18:54:39.703 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.703 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 218, in send 2023-09-06T18:54:39.704 INFO:teuthology.orchestra.run.smithi099.stderr: return super().send(str) 2023-09-06T18:54:39.704 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.704 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.704 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 205, in connect 2023-09-06T18:54:39.704 INFO:teuthology.orchestra.run.smithi099.stderr: conn = self._new_conn() 2023-09-06T18:54:39.705 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 187, in _new_conn 2023-09-06T18:54:39.705 INFO:teuthology.orchestra.run.smithi099.stderr: self, "Failed to establish a new connection: %s" % e 2023-09-06T18:54:39.705 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused 2023-09-06T18:54:39.705 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.705 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.706 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.706 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.706 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 278, in _do_get_response 2023-09-06T18:54:39.706 INFO:teuthology.orchestra.run.smithi099.stderr: http_response = self._send(request) 2023-09-06T18:54:39.706 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 374, in _send 2023-09-06T18:54:39.707 INFO:teuthology.orchestra.run.smithi099.stderr: return self.http_session.send(request) 2023-09-06T18:54:39.707 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 472, in send 2023-09-06T18:54:39.707 INFO:teuthology.orchestra.run.smithi099.stderr: raise EndpointConnectionError(endpoint_url=request.url, error=e) 2023-09-06T18:54:39.707 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://smithi099.front.sepia.ceph.com:80/cjhpse-4" 2023-09-06T18:54:39.707 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event needs-retry.s3.CreateBucket: calling handler 2023-09-06T18:54:39.707 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.retryhandler: DEBUG: retry needed, retryable exception caught: Could not connect to the endpoint URL: "http://smithi099.front.sepia.ceph.com:80/cjhpse-4" 2023-09-06T18:54:39.708 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.708 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 175, in _new_conn 2023-09-06T18:54:39.708 INFO:teuthology.orchestra.run.smithi099.stderr: (self._dns_host, self.port), self.timeout, **extra_kw 2023-09-06T18:54:39.708 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 95, in create_connection 2023-09-06T18:54:39.708 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.709 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 85, in create_connection 2023-09-06T18:54:39.709 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.709 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.709 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.709 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.710 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.710 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.710 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 452, in send 2023-09-06T18:54:39.710 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=self._chunked(request.headers), 2023-09-06T18:54:39.710 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 799, in urlopen 2023-09-06T18:54:39.710 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] 2023-09-06T18:54:39.711 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/retry.py", line 525, in increment 2023-09-06T18:54:39.711 INFO:teuthology.orchestra.run.smithi099.stderr: raise six.reraise(type(error), error, _stacktrace) 2023-09-06T18:54:39.711 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/packages/six.py", line 770, in reraise 2023-09-06T18:54:39.711 INFO:teuthology.orchestra.run.smithi099.stderr: raise value 2023-09-06T18:54:39.711 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 721, in urlopen 2023-09-06T18:54:39.712 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=chunked, 2023-09-06T18:54:39.712 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 415, in _make_request 2023-09-06T18:54:39.712 INFO:teuthology.orchestra.run.smithi099.stderr: conn.request(method, url, **httplib_request_kw) 2023-09-06T18:54:39.712 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 244, in request 2023-09-06T18:54:39.712 INFO:teuthology.orchestra.run.smithi099.stderr: super(HTTPConnection, self).request(method, url, body=body, headers=headers) 2023-09-06T18:54:39.713 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.713 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.713 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 95, in _send_request 2023-09-06T18:54:39.713 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, body, headers, *args, **kwargs 2023-09-06T18:54:39.714 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.714 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.714 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.714 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.714 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 123, in _send_output 2023-09-06T18:54:39.715 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.715 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 218, in send 2023-09-06T18:54:39.715 INFO:teuthology.orchestra.run.smithi099.stderr: return super().send(str) 2023-09-06T18:54:39.715 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.715 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.716 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 205, in connect 2023-09-06T18:54:39.716 INFO:teuthology.orchestra.run.smithi099.stderr: conn = self._new_conn() 2023-09-06T18:54:39.716 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 187, in _new_conn 2023-09-06T18:54:39.716 INFO:teuthology.orchestra.run.smithi099.stderr: self, "Failed to establish a new connection: %s" % e 2023-09-06T18:54:39.716 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused 2023-09-06T18:54:39.717 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.717 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.717 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.717 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.717 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 308, in _should_retry 2023-09-06T18:54:39.718 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, response, caught_exception 2023-09-06T18:54:39.718 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 364, in __call__ 2023-09-06T18:54:39.718 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, response, caught_exception 2023-09-06T18:54:39.718 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 248, in __call__ 2023-09-06T18:54:39.718 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, caught_exception 2023-09-06T18:54:39.719 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 416, in _check_caught_exception 2023-09-06T18:54:39.719 INFO:teuthology.orchestra.run.smithi099.stderr: raise caught_exception 2023-09-06T18:54:39.719 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 278, in _do_get_response 2023-09-06T18:54:39.719 INFO:teuthology.orchestra.run.smithi099.stderr: http_response = self._send(request) 2023-09-06T18:54:39.719 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 374, in _send 2023-09-06T18:54:39.720 INFO:teuthology.orchestra.run.smithi099.stderr: return self.http_session.send(request) 2023-09-06T18:54:39.720 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 472, in send 2023-09-06T18:54:39.720 INFO:teuthology.orchestra.run.smithi099.stderr: raise EndpointConnectionError(endpoint_url=request.url, error=e) 2023-09-06T18:54:39.720 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://smithi099.front.sepia.ceph.com:80/cjhpse-4" 2023-09-06T18:54:39.720 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.retryhandler: DEBUG: Retry needed, action of: 1.844342081654472 2023-09-06T18:54:39.721 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event needs-retry.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.721 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Response received to retry, sleeping for 1.844342081654472 seconds 2023-09-06T18:54:39.721 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.721 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.722 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.722 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.722 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.722 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.722 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.723 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1398.141730415 and callback=>; now=1368.141730415; delay=30.0 2023-09-06T18:54:39.723 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.723 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.723 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1398.15362622 and callback=>; now=1368.15362622; delay=30.0 2023-09-06T18:54:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.726 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.726 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.726 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.726 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.726 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler 2023-09-06T18:54:39.727 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-sign.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.727 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Calculating signature using v4 auth. 2023-09-06T18:54:39.727 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: CanonicalRequest: 2023-09-06T18:54:39.727 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.727 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-4 2023-09-06T18:54:39.728 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.728 INFO:teuthology.orchestra.run.smithi099.stderr:host:smithi099.front.sepia.ceph.com 2023-09-06T18:54:39.728 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-acl:public-read-write 2023-09-06T18:54:39.728 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2023-09-06T18:54:39.728 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-date:20230906T184541Z 2023-09-06T18:54:39.729 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.729 INFO:teuthology.orchestra.run.smithi099.stderr:host;x-amz-acl;x-amz-content-sha256;x-amz-date 2023-09-06T18:54:39.729 INFO:teuthology.orchestra.run.smithi099.stderr:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2023-09-06T18:54:39.729 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: StringToSign: 2023-09-06T18:54:39.729 INFO:teuthology.orchestra.run.smithi099.stderr:AWS4-HMAC-SHA256 2023-09-06T18:54:39.729 INFO:teuthology.orchestra.run.smithi099.stderr:20230906T184541Z 2023-09-06T18:54:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:20230906/us-east-1/s3/aws4_request 2023-09-06T18:54:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:d582d6938557488172076e104167aad7594a17bfb64ee8ec7bdb2b0417052b06 2023-09-06T18:54:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Signature: 2023-09-06T18:54:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:c606e56298db3a14cc25163e0e27946e5e1fd30e96776a10fe2634663cd93804 2023-09-06T18:54:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.s3.CreateBucket: calling handler 2023-09-06T18:54:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Sending http request: 2023-09-06T18:54:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.connectionpool: DEBUG: Starting new HTTP connection (3): smithi099.front.sepia.ceph.com:80 2023-09-06T18:54:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Exception received when sending HTTP request. 2023-09-06T18:54:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.731 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 175, in _new_conn 2023-09-06T18:54:39.732 INFO:teuthology.orchestra.run.smithi099.stderr: (self._dns_host, self.port), self.timeout, **extra_kw 2023-09-06T18:54:39.732 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 95, in create_connection 2023-09-06T18:54:39.732 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.732 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 85, in create_connection 2023-09-06T18:54:39.732 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.733 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.733 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.733 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 452, in send 2023-09-06T18:54:39.734 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=self._chunked(request.headers), 2023-09-06T18:54:39.734 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 799, in urlopen 2023-09-06T18:54:39.734 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] 2023-09-06T18:54:39.734 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/retry.py", line 525, in increment 2023-09-06T18:54:39.734 INFO:teuthology.orchestra.run.smithi099.stderr: raise six.reraise(type(error), error, _stacktrace) 2023-09-06T18:54:39.735 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/packages/six.py", line 770, in reraise 2023-09-06T18:54:39.735 INFO:teuthology.orchestra.run.smithi099.stderr: raise value 2023-09-06T18:54:39.735 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 721, in urlopen 2023-09-06T18:54:39.735 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=chunked, 2023-09-06T18:54:39.735 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 415, in _make_request 2023-09-06T18:54:39.735 INFO:teuthology.orchestra.run.smithi099.stderr: conn.request(method, url, **httplib_request_kw) 2023-09-06T18:54:39.736 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 244, in request 2023-09-06T18:54:39.736 INFO:teuthology.orchestra.run.smithi099.stderr: super(HTTPConnection, self).request(method, url, body=body, headers=headers) 2023-09-06T18:54:39.736 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.736 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.736 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 95, in _send_request 2023-09-06T18:54:39.737 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, body, headers, *args, **kwargs 2023-09-06T18:54:39.737 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.737 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.737 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.737 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.738 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 123, in _send_output 2023-09-06T18:54:39.738 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.738 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 218, in send 2023-09-06T18:54:39.738 INFO:teuthology.orchestra.run.smithi099.stderr: return super().send(str) 2023-09-06T18:54:39.738 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.738 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.739 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 205, in connect 2023-09-06T18:54:39.739 INFO:teuthology.orchestra.run.smithi099.stderr: conn = self._new_conn() 2023-09-06T18:54:39.739 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 187, in _new_conn 2023-09-06T18:54:39.739 INFO:teuthology.orchestra.run.smithi099.stderr: self, "Failed to establish a new connection: %s" % e 2023-09-06T18:54:39.740 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused 2023-09-06T18:54:39.740 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.740 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.740 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.740 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.741 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 278, in _do_get_response 2023-09-06T18:54:39.741 INFO:teuthology.orchestra.run.smithi099.stderr: http_response = self._send(request) 2023-09-06T18:54:39.741 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 374, in _send 2023-09-06T18:54:39.741 INFO:teuthology.orchestra.run.smithi099.stderr: return self.http_session.send(request) 2023-09-06T18:54:39.741 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 472, in send 2023-09-06T18:54:39.742 INFO:teuthology.orchestra.run.smithi099.stderr: raise EndpointConnectionError(endpoint_url=request.url, error=e) 2023-09-06T18:54:39.742 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://smithi099.front.sepia.ceph.com:80/cjhpse-4" 2023-09-06T18:54:39.742 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event needs-retry.s3.CreateBucket: calling handler 2023-09-06T18:54:39.742 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.retryhandler: DEBUG: retry needed, retryable exception caught: Could not connect to the endpoint URL: "http://smithi099.front.sepia.ceph.com:80/cjhpse-4" 2023-09-06T18:54:39.742 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.743 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 175, in _new_conn 2023-09-06T18:54:39.743 INFO:teuthology.orchestra.run.smithi099.stderr: (self._dns_host, self.port), self.timeout, **extra_kw 2023-09-06T18:54:39.743 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 95, in create_connection 2023-09-06T18:54:39.743 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.743 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 85, in create_connection 2023-09-06T18:54:39.743 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.744 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.744 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.745 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 452, in send 2023-09-06T18:54:39.745 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=self._chunked(request.headers), 2023-09-06T18:54:39.745 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 799, in urlopen 2023-09-06T18:54:39.745 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] 2023-09-06T18:54:39.745 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/retry.py", line 525, in increment 2023-09-06T18:54:39.746 INFO:teuthology.orchestra.run.smithi099.stderr: raise six.reraise(type(error), error, _stacktrace) 2023-09-06T18:54:39.746 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/packages/six.py", line 770, in reraise 2023-09-06T18:54:39.746 INFO:teuthology.orchestra.run.smithi099.stderr: raise value 2023-09-06T18:54:39.746 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 721, in urlopen 2023-09-06T18:54:39.746 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=chunked, 2023-09-06T18:54:39.747 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 415, in _make_request 2023-09-06T18:54:39.747 INFO:teuthology.orchestra.run.smithi099.stderr: conn.request(method, url, **httplib_request_kw) 2023-09-06T18:54:39.747 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 244, in request 2023-09-06T18:54:39.747 INFO:teuthology.orchestra.run.smithi099.stderr: super(HTTPConnection, self).request(method, url, body=body, headers=headers) 2023-09-06T18:54:39.747 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.748 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.748 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 95, in _send_request 2023-09-06T18:54:39.748 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, body, headers, *args, **kwargs 2023-09-06T18:54:39.748 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.748 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.748 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.749 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.749 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 123, in _send_output 2023-09-06T18:54:39.749 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.749 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 218, in send 2023-09-06T18:54:39.749 INFO:teuthology.orchestra.run.smithi099.stderr: return super().send(str) 2023-09-06T18:54:39.750 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.750 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.750 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 205, in connect 2023-09-06T18:54:39.750 INFO:teuthology.orchestra.run.smithi099.stderr: conn = self._new_conn() 2023-09-06T18:54:39.750 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 187, in _new_conn 2023-09-06T18:54:39.751 INFO:teuthology.orchestra.run.smithi099.stderr: self, "Failed to establish a new connection: %s" % e 2023-09-06T18:54:39.751 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused 2023-09-06T18:54:39.751 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.751 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.751 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.752 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.752 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 308, in _should_retry 2023-09-06T18:54:39.752 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, response, caught_exception 2023-09-06T18:54:39.752 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 364, in __call__ 2023-09-06T18:54:39.752 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, response, caught_exception 2023-09-06T18:54:39.752 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 248, in __call__ 2023-09-06T18:54:39.753 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, caught_exception 2023-09-06T18:54:39.753 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 416, in _check_caught_exception 2023-09-06T18:54:39.753 INFO:teuthology.orchestra.run.smithi099.stderr: raise caught_exception 2023-09-06T18:54:39.753 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 278, in _do_get_response 2023-09-06T18:54:39.753 INFO:teuthology.orchestra.run.smithi099.stderr: http_response = self._send(request) 2023-09-06T18:54:39.754 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 374, in _send 2023-09-06T18:54:39.754 INFO:teuthology.orchestra.run.smithi099.stderr: return self.http_session.send(request) 2023-09-06T18:54:39.754 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 472, in send 2023-09-06T18:54:39.754 INFO:teuthology.orchestra.run.smithi099.stderr: raise EndpointConnectionError(endpoint_url=request.url, error=e) 2023-09-06T18:54:39.754 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://smithi099.front.sepia.ceph.com:80/cjhpse-4" 2023-09-06T18:54:39.754 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.retryhandler: DEBUG: Retry needed, action of: 0.4050868941628032 2023-09-06T18:54:39.755 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event needs-retry.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.755 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Response received to retry, sleeping for 0.4050868941628032 seconds 2023-09-06T18:54:39.755 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.755 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.756 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.756 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler 2023-09-06T18:54:39.756 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-sign.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.756 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Calculating signature using v4 auth. 2023-09-06T18:54:39.756 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: CanonicalRequest: 2023-09-06T18:54:39.757 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.757 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-4 2023-09-06T18:54:39.757 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.757 INFO:teuthology.orchestra.run.smithi099.stderr:host:smithi099.front.sepia.ceph.com 2023-09-06T18:54:39.757 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-acl:public-read-write 2023-09-06T18:54:39.758 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2023-09-06T18:54:39.758 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-date:20230906T184541Z 2023-09-06T18:54:39.758 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.758 INFO:teuthology.orchestra.run.smithi099.stderr:host;x-amz-acl;x-amz-content-sha256;x-amz-date 2023-09-06T18:54:39.758 INFO:teuthology.orchestra.run.smithi099.stderr:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2023-09-06T18:54:39.758 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: StringToSign: 2023-09-06T18:54:39.759 INFO:teuthology.orchestra.run.smithi099.stderr:AWS4-HMAC-SHA256 2023-09-06T18:54:39.759 INFO:teuthology.orchestra.run.smithi099.stderr:20230906T184541Z 2023-09-06T18:54:39.759 INFO:teuthology.orchestra.run.smithi099.stderr:20230906/us-east-1/s3/aws4_request 2023-09-06T18:54:39.759 INFO:teuthology.orchestra.run.smithi099.stderr:d582d6938557488172076e104167aad7594a17bfb64ee8ec7bdb2b0417052b06 2023-09-06T18:54:39.759 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Signature: 2023-09-06T18:54:39.760 INFO:teuthology.orchestra.run.smithi099.stderr:c606e56298db3a14cc25163e0e27946e5e1fd30e96776a10fe2634663cd93804 2023-09-06T18:54:39.760 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.s3.CreateBucket: calling handler 2023-09-06T18:54:39.760 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Sending http request: 2023-09-06T18:54:39.760 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.connectionpool: DEBUG: Starting new HTTP connection (4): smithi099.front.sepia.ceph.com:80 2023-09-06T18:54:39.760 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Exception received when sending HTTP request. 2023-09-06T18:54:39.761 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.761 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 175, in _new_conn 2023-09-06T18:54:39.761 INFO:teuthology.orchestra.run.smithi099.stderr: (self._dns_host, self.port), self.timeout, **extra_kw 2023-09-06T18:54:39.761 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 95, in create_connection 2023-09-06T18:54:39.761 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.762 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 85, in create_connection 2023-09-06T18:54:39.762 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.762 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.762 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.762 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.763 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.763 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.763 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 452, in send 2023-09-06T18:54:39.763 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=self._chunked(request.headers), 2023-09-06T18:54:39.763 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 799, in urlopen 2023-09-06T18:54:39.764 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] 2023-09-06T18:54:39.764 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/retry.py", line 525, in increment 2023-09-06T18:54:39.764 INFO:teuthology.orchestra.run.smithi099.stderr: raise six.reraise(type(error), error, _stacktrace) 2023-09-06T18:54:39.764 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/packages/six.py", line 770, in reraise 2023-09-06T18:54:39.764 INFO:teuthology.orchestra.run.smithi099.stderr: raise value 2023-09-06T18:54:39.764 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 721, in urlopen 2023-09-06T18:54:39.765 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=chunked, 2023-09-06T18:54:39.765 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 415, in _make_request 2023-09-06T18:54:39.765 INFO:teuthology.orchestra.run.smithi099.stderr: conn.request(method, url, **httplib_request_kw) 2023-09-06T18:54:39.765 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 244, in request 2023-09-06T18:54:39.765 INFO:teuthology.orchestra.run.smithi099.stderr: super(HTTPConnection, self).request(method, url, body=body, headers=headers) 2023-09-06T18:54:39.766 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.766 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.766 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 95, in _send_request 2023-09-06T18:54:39.766 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, body, headers, *args, **kwargs 2023-09-06T18:54:39.766 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.767 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.767 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.767 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.767 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 123, in _send_output 2023-09-06T18:54:39.767 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.768 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 218, in send 2023-09-06T18:54:39.768 INFO:teuthology.orchestra.run.smithi099.stderr: return super().send(str) 2023-09-06T18:54:39.768 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.768 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.768 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 205, in connect 2023-09-06T18:54:39.768 INFO:teuthology.orchestra.run.smithi099.stderr: conn = self._new_conn() 2023-09-06T18:54:39.769 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 187, in _new_conn 2023-09-06T18:54:39.769 INFO:teuthology.orchestra.run.smithi099.stderr: self, "Failed to establish a new connection: %s" % e 2023-09-06T18:54:39.769 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused 2023-09-06T18:54:39.769 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.769 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.770 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.770 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.770 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 278, in _do_get_response 2023-09-06T18:54:39.770 INFO:teuthology.orchestra.run.smithi099.stderr: http_response = self._send(request) 2023-09-06T18:54:39.770 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 374, in _send 2023-09-06T18:54:39.771 INFO:teuthology.orchestra.run.smithi099.stderr: return self.http_session.send(request) 2023-09-06T18:54:39.771 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 472, in send 2023-09-06T18:54:39.771 INFO:teuthology.orchestra.run.smithi099.stderr: raise EndpointConnectionError(endpoint_url=request.url, error=e) 2023-09-06T18:54:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://smithi099.front.sepia.ceph.com:80/cjhpse-4" 2023-09-06T18:54:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event needs-retry.s3.CreateBucket: calling handler 2023-09-06T18:54:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.retryhandler: DEBUG: retry needed, retryable exception caught: Could not connect to the endpoint URL: "http://smithi099.front.sepia.ceph.com:80/cjhpse-4" 2023-09-06T18:54:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.772 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 175, in _new_conn 2023-09-06T18:54:39.772 INFO:teuthology.orchestra.run.smithi099.stderr: (self._dns_host, self.port), self.timeout, **extra_kw 2023-09-06T18:54:39.772 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 95, in create_connection 2023-09-06T18:54:39.772 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.773 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 85, in create_connection 2023-09-06T18:54:39.773 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.773 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.774 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.774 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.774 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.774 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 452, in send 2023-09-06T18:54:39.774 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=self._chunked(request.headers), 2023-09-06T18:54:39.775 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 799, in urlopen 2023-09-06T18:54:39.775 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] 2023-09-06T18:54:39.775 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/retry.py", line 525, in increment 2023-09-06T18:54:39.775 INFO:teuthology.orchestra.run.smithi099.stderr: raise six.reraise(type(error), error, _stacktrace) 2023-09-06T18:54:39.775 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/packages/six.py", line 770, in reraise 2023-09-06T18:54:39.776 INFO:teuthology.orchestra.run.smithi099.stderr: raise value 2023-09-06T18:54:39.776 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 721, in urlopen 2023-09-06T18:54:39.776 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=chunked, 2023-09-06T18:54:39.776 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 415, in _make_request 2023-09-06T18:54:39.776 INFO:teuthology.orchestra.run.smithi099.stderr: conn.request(method, url, **httplib_request_kw) 2023-09-06T18:54:39.777 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 244, in request 2023-09-06T18:54:39.777 INFO:teuthology.orchestra.run.smithi099.stderr: super(HTTPConnection, self).request(method, url, body=body, headers=headers) 2023-09-06T18:54:39.777 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.777 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.777 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 95, in _send_request 2023-09-06T18:54:39.778 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, body, headers, *args, **kwargs 2023-09-06T18:54:39.778 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.778 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.778 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.778 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.779 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 123, in _send_output 2023-09-06T18:54:39.779 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.779 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 218, in send 2023-09-06T18:54:39.779 INFO:teuthology.orchestra.run.smithi099.stderr: return super().send(str) 2023-09-06T18:54:39.779 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.780 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.780 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 205, in connect 2023-09-06T18:54:39.780 INFO:teuthology.orchestra.run.smithi099.stderr: conn = self._new_conn() 2023-09-06T18:54:39.780 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 187, in _new_conn 2023-09-06T18:54:39.780 INFO:teuthology.orchestra.run.smithi099.stderr: self, "Failed to establish a new connection: %s" % e 2023-09-06T18:54:39.781 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused 2023-09-06T18:54:39.781 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.781 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.781 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.781 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.781 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 308, in _should_retry 2023-09-06T18:54:39.782 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, response, caught_exception 2023-09-06T18:54:39.782 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 364, in __call__ 2023-09-06T18:54:39.785 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, response, caught_exception 2023-09-06T18:54:39.785 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 248, in __call__ 2023-09-06T18:54:39.785 INFO:teuthology.orchestra.run.smithi099.stderr: attempt_number, caught_exception 2023-09-06T18:54:39.785 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 416, in _check_caught_exception 2023-09-06T18:54:39.785 INFO:teuthology.orchestra.run.smithi099.stderr: raise caught_exception 2023-09-06T18:54:39.786 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 278, in _do_get_response 2023-09-06T18:54:39.786 INFO:teuthology.orchestra.run.smithi099.stderr: http_response = self._send(request) 2023-09-06T18:54:39.786 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 374, in _send 2023-09-06T18:54:39.786 INFO:teuthology.orchestra.run.smithi099.stderr: return self.http_session.send(request) 2023-09-06T18:54:39.786 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 472, in send 2023-09-06T18:54:39.787 INFO:teuthology.orchestra.run.smithi099.stderr: raise EndpointConnectionError(endpoint_url=request.url, error=e) 2023-09-06T18:54:39.787 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://smithi099.front.sepia.ceph.com:80/cjhpse-4" 2023-09-06T18:54:39.787 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.retryhandler: DEBUG: Retry needed, action of: 6.435967607392229 2023-09-06T18:54:39.787 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event needs-retry.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.788 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Response received to retry, sleeping for 6.435967607392229 seconds 2023-09-06T18:54:39.788 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.788 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.788 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.788 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler 2023-09-06T18:54:39.789 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event before-sign.s3.CreateBucket: calling handler > 2023-09-06T18:54:39.789 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Calculating signature using v4 auth. 2023-09-06T18:54:39.789 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: CanonicalRequest: 2023-09-06T18:54:39.789 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.789 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-4 2023-09-06T18:54:39.790 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.790 INFO:teuthology.orchestra.run.smithi099.stderr:host:smithi099.front.sepia.ceph.com 2023-09-06T18:54:39.790 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-acl:public-read-write 2023-09-06T18:54:39.790 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2023-09-06T18:54:39.790 INFO:teuthology.orchestra.run.smithi099.stderr:x-amz-date:20230906T184547Z 2023-09-06T18:54:39.791 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.791 INFO:teuthology.orchestra.run.smithi099.stderr:host;x-amz-acl;x-amz-content-sha256;x-amz-date 2023-09-06T18:54:39.791 INFO:teuthology.orchestra.run.smithi099.stderr:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2023-09-06T18:54:39.791 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: StringToSign: 2023-09-06T18:54:39.791 INFO:teuthology.orchestra.run.smithi099.stderr:AWS4-HMAC-SHA256 2023-09-06T18:54:39.792 INFO:teuthology.orchestra.run.smithi099.stderr:20230906T184547Z 2023-09-06T18:54:39.792 INFO:teuthology.orchestra.run.smithi099.stderr:20230906/us-east-1/s3/aws4_request 2023-09-06T18:54:39.792 INFO:teuthology.orchestra.run.smithi099.stderr:44d0e9d3fea0c67430133f10593bc20f21c845084bb523c340740d236dab03ca 2023-09-06T18:54:39.792 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.auth: DEBUG: Signature: 2023-09-06T18:54:39.793 INFO:teuthology.orchestra.run.smithi099.stderr:7fe9e18fa227d55ae572bdbdee0fac48edfc02877485973ee0b934f4778c57db 2023-09-06T18:54:39.793 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event request-created.s3.CreateBucket: calling handler 2023-09-06T18:54:39.793 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Sending http request: 2023-09-06T18:54:39.793 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.connectionpool: DEBUG: Starting new HTTP connection (5): smithi099.front.sepia.ceph.com:80 2023-09-06T18:54:39.793 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.endpoint: DEBUG: Exception received when sending HTTP request. 2023-09-06T18:54:39.794 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.794 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 175, in _new_conn 2023-09-06T18:54:39.794 INFO:teuthology.orchestra.run.smithi099.stderr: (self._dns_host, self.port), self.timeout, **extra_kw 2023-09-06T18:54:39.794 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 95, in create_connection 2023-09-06T18:54:39.794 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.795 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/connection.py", line 85, in create_connection 2023-09-06T18:54:39.795 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.795 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.795 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.795 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.796 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.796 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.796 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 452, in send 2023-09-06T18:54:39.796 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=self._chunked(request.headers), 2023-09-06T18:54:39.796 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 799, in urlopen 2023-09-06T18:54:39.797 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] 2023-09-06T18:54:39.797 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/util/retry.py", line 525, in increment 2023-09-06T18:54:39.797 INFO:teuthology.orchestra.run.smithi099.stderr: raise six.reraise(type(error), error, _stacktrace) 2023-09-06T18:54:39.797 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/packages/six.py", line 770, in reraise 2023-09-06T18:54:39.797 INFO:teuthology.orchestra.run.smithi099.stderr: raise value 2023-09-06T18:54:39.798 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 721, in urlopen 2023-09-06T18:54:39.798 INFO:teuthology.orchestra.run.smithi099.stderr: chunked=chunked, 2023-09-06T18:54:39.798 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 415, in _make_request 2023-09-06T18:54:39.798 INFO:teuthology.orchestra.run.smithi099.stderr: conn.request(method, url, **httplib_request_kw) 2023-09-06T18:54:39.798 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 244, in request 2023-09-06T18:54:39.799 INFO:teuthology.orchestra.run.smithi099.stderr: super(HTTPConnection, self).request(method, url, body=body, headers=headers) 2023-09-06T18:54:39.799 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.799 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.799 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 95, in _send_request 2023-09-06T18:54:39.799 INFO:teuthology.orchestra.run.smithi099.stderr: method, url, body, headers, *args, **kwargs 2023-09-06T18:54:39.800 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.800 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.800 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.800 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.800 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 123, in _send_output 2023-09-06T18:54:39.801 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.801 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/awsrequest.py", line 218, in send 2023-09-06T18:54:39.801 INFO:teuthology.orchestra.run.smithi099.stderr: return super().send(str) 2023-09-06T18:54:39.801 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.801 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.801 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 205, in connect 2023-09-06T18:54:39.802 INFO:teuthology.orchestra.run.smithi099.stderr: conn = self._new_conn() 2023-09-06T18:54:39.802 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/urllib3/connection.py", line 187, in _new_conn 2023-09-06T18:54:39.802 INFO:teuthology.orchestra.run.smithi099.stderr: self, "Failed to establish a new connection: %s" % e 2023-09-06T18:54:39.802 INFO:teuthology.orchestra.run.smithi099.stderr:urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused 2023-09-06T18:54:39.802 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.803 INFO:teuthology.orchestra.run.smithi099.stderr:During handling of the above exception, another exception occurred: 2023-09-06T18:54:39.803 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.803 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.803 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 278, in _do_get_response 2023-09-06T18:54:39.803 INFO:teuthology.orchestra.run.smithi099.stderr: http_response = self._send(request) 2023-09-06T18:54:39.804 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/endpoint.py", line 374, in _send 2023-09-06T18:54:39.804 INFO:teuthology.orchestra.run.smithi099.stderr: return self.http_session.send(request) 2023-09-06T18:54:39.804 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/botocore/httpsession.py", line 472, in send 2023-09-06T18:54:39.804 INFO:teuthology.orchestra.run.smithi099.stderr: raise EndpointConnectionError(endpoint_url=request.url, error=e) 2023-09-06T18:54:39.804 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://smithi099.front.sepia.ceph.com:80/cjhpse-4" 2023-09-06T18:54:39.805 INFO:teuthology.orchestra.run.smithi099.stderr:botocore.hooks: DEBUG: Event needs-retry.s3.CreateBucket: calling handler 2023-09-06T18:54:39.805 INFO:teuthology.orchestra.run.smithi099.stderr:--------------------- >> end captured logging << --------------------- 2023-09-06T18:54:39.805 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.805 INFO:teuthology.orchestra.run.smithi099.stderr:====================================================================== 2023-09-06T18:54:39.805 INFO:teuthology.orchestra.run.smithi099.stderr:ERROR: test multipart object upload on master 2023-09-06T18:54:39.806 INFO:teuthology.orchestra.run.smithi099.stderr:---------------------------------------------------------------------- 2023-09-06T18:54:39.806 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.806 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest 2023-09-06T18:54:39.806 INFO:teuthology.orchestra.run.smithi099.stderr: self.test(*self.arg) 2023-09-06T18:54:39.806 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 2362, in test_ps_s3_multipart_on_master 2023-09-06T18:54:39.806 INFO:teuthology.orchestra.run.smithi099.stderr: bucket = conn.create_bucket(bucket_name) 2023-09-06T18:54:39.807 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 619, in create_bucket 2023-09-06T18:54:39.807 INFO:teuthology.orchestra.run.smithi099.stderr: data=data) 2023-09-06T18:54:39.807 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 671, in make_request 2023-09-06T18:54:39.807 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler 2023-09-06T18:54:39.807 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1071, in make_request 2023-09-06T18:54:39.808 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler) 2023-09-06T18:54:39.808 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1030, in _mexe 2023-09-06T18:54:39.808 INFO:teuthology.orchestra.run.smithi099.stderr: raise ex 2023-09-06T18:54:39.808 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 943, in _mexe 2023-09-06T18:54:39.808 INFO:teuthology.orchestra.run.smithi099.stderr: request.body, request.headers) 2023-09-06T18:54:39.809 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.809 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.809 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.809 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.809 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.809 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.810 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1044, in _send_output 2023-09-06T18:54:39.810 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.810 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.810 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.810 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 954, in connect 2023-09-06T18:54:39.811 INFO:teuthology.orchestra.run.smithi099.stderr: (self.host,self.port), self.timeout, self.source_address) 2023-09-06T18:54:39.811 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 724, in create_connection 2023-09-06T18:54:39.811 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.811 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 713, in create_connection 2023-09-06T18:54:39.812 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.812 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.812 INFO:teuthology.orchestra.run.smithi099.stderr:-------------------- >> begin captured logging << -------------------- 2023-09-06T18:54:39.812 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using access key provided by client. 2023-09-06T18:54:39.812 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using secret key provided by client. 2023-09-06T18:54:39.812 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-5/ 2023-09-06T18:54:39.813 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-5/ 2023-09-06T18:54:39.813 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:39.813 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-5/ 2023-09-06T18:54:39.813 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:39.813 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {} 2023-09-06T18:54:39.814 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:39.814 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:39.814 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:39.814 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.814 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.815 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.815 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:45:47 GMT 2023-09-06T18:54:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-5/ 2023-09-06T18:54:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:m/1TNnkYthIQtY9ex8B6MqVVP2o= 2023-09-06T18:54:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:45:47 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:m/1TNnkYthIQtY9ex8B6MqVVP2o=', 'Content-Length': '0'} 2023-09-06T18:54:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.817 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.817 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.817 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.817 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.818 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.818 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.818 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:45:47 GMT 2023-09-06T18:54:39.818 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-5/ 2023-09-06T18:54:39.819 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.819 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:m/1TNnkYthIQtY9ex8B6MqVVP2o= 2023-09-06T18:54:39.819 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:45:47 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:m/1TNnkYthIQtY9ex8B6MqVVP2o=', 'Content-Length': '0'} 2023-09-06T18:54:39.819 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.819 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.820 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.820 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.820 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.820 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.820 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.821 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:45:47 GMT 2023-09-06T18:54:39.821 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-5/ 2023-09-06T18:54:39.821 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.821 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:m/1TNnkYthIQtY9ex8B6MqVVP2o= 2023-09-06T18:54:39.821 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:45:47 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:m/1TNnkYthIQtY9ex8B6MqVVP2o=', 'Content-Length': '0'} 2023-09-06T18:54:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.823 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.823 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.823 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:45:47 GMT 2023-09-06T18:54:39.823 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-5/ 2023-09-06T18:54:39.823 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.824 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:m/1TNnkYthIQtY9ex8B6MqVVP2o= 2023-09-06T18:54:39.824 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:45:47 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:m/1TNnkYthIQtY9ex8B6MqVVP2o=', 'Content-Length': '0'} 2023-09-06T18:54:39.824 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.824 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.824 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.825 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.825 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.825 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.825 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.825 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:45:47 GMT 2023-09-06T18:54:39.826 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-5/ 2023-09-06T18:54:39.826 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.826 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:m/1TNnkYthIQtY9ex8B6MqVVP2o= 2023-09-06T18:54:39.826 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:45:47 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:m/1TNnkYthIQtY9ex8B6MqVVP2o=', 'Content-Length': '0'} 2023-09-06T18:54:39.826 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.827 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.827 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.827 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.827 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.827 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.828 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.828 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:45:47 GMT 2023-09-06T18:54:39.828 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-5/ 2023-09-06T18:54:39.828 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.828 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:m/1TNnkYthIQtY9ex8B6MqVVP2o= 2023-09-06T18:54:39.829 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:45:47 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:m/1TNnkYthIQtY9ex8B6MqVVP2o=', 'Content-Length': '0'} 2023-09-06T18:54:39.829 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.829 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.829 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.829 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.830 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.830 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.830 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.830 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.830 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1428.142664145 and callback=>; now=1398.142664145; delay=30.0 2023-09-06T18:54:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1428.154404108 and callback=>; now=1398.154404108; delay=30.0 2023-09-06T18:54:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.834 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.834 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.834 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.834 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.834 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:45:47 GMT 2023-09-06T18:54:39.835 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-5/ 2023-09-06T18:54:39.835 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.835 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:m/1TNnkYthIQtY9ex8B6MqVVP2o= 2023-09-06T18:54:39.835 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:45:47 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:m/1TNnkYthIQtY9ex8B6MqVVP2o=', 'Content-Length': '0'} 2023-09-06T18:54:39.835 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.836 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.836 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 10 heartbeat frames, sent 10, idle intervals 0 2023-09-06T18:54:39.836 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1488.161867143 and callback=>; now=1423.161867143; delay=65 2023-09-06T18:54:39.836 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 10 heartbeat frames, sent 10, idle intervals 0 2023-09-06T18:54:39.836 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1488.17244465 and callback=>; now=1423.17244465; delay=65 2023-09-06T18:54:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:--------------------- >> end captured logging << --------------------- 2023-09-06T18:54:39.837 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:====================================================================== 2023-09-06T18:54:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:ERROR: test s3 notification of metadata on master 2023-09-06T18:54:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:---------------------------------------------------------------------- 2023-09-06T18:54:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.838 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest 2023-09-06T18:54:39.838 INFO:teuthology.orchestra.run.smithi099.stderr: self.test(*self.arg) 2023-09-06T18:54:39.838 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 2451, in test_ps_s3_metadata_filter_on_master 2023-09-06T18:54:39.838 INFO:teuthology.orchestra.run.smithi099.stderr: bucket = conn.create_bucket(bucket_name) 2023-09-06T18:54:39.838 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 619, in create_bucket 2023-09-06T18:54:39.839 INFO:teuthology.orchestra.run.smithi099.stderr: data=data) 2023-09-06T18:54:39.839 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 671, in make_request 2023-09-06T18:54:39.839 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler 2023-09-06T18:54:39.839 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1071, in make_request 2023-09-06T18:54:39.839 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler) 2023-09-06T18:54:39.840 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1030, in _mexe 2023-09-06T18:54:39.840 INFO:teuthology.orchestra.run.smithi099.stderr: raise ex 2023-09-06T18:54:39.840 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 943, in _mexe 2023-09-06T18:54:39.840 INFO:teuthology.orchestra.run.smithi099.stderr: request.body, request.headers) 2023-09-06T18:54:39.840 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.841 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.841 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.841 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.841 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.841 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.841 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1044, in _send_output 2023-09-06T18:54:39.842 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.842 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.842 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.842 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 954, in connect 2023-09-06T18:54:39.842 INFO:teuthology.orchestra.run.smithi099.stderr: (self.host,self.port), self.timeout, self.source_address) 2023-09-06T18:54:39.843 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 724, in create_connection 2023-09-06T18:54:39.843 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.843 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 713, in create_connection 2023-09-06T18:54:39.843 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.844 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.844 INFO:teuthology.orchestra.run.smithi099.stderr:-------------------- >> begin captured logging << -------------------- 2023-09-06T18:54:39.844 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using access key provided by client. 2023-09-06T18:54:39.844 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using secret key provided by client. 2023-09-06T18:54:39.844 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-6/ 2023-09-06T18:54:39.845 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-6/ 2023-09-06T18:54:39.845 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:39.845 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-6/ 2023-09-06T18:54:39.845 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:39.845 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {} 2023-09-06T18:54:39.845 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:39.846 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:39.846 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:39.846 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.846 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.846 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.847 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.847 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.847 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.847 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:46:38 GMT 2023-09-06T18:54:39.847 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-6/ 2023-09-06T18:54:39.848 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.848 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:cf+LtMLOAFrqmfMpz7xFqBbpwDE= 2023-09-06T18:54:39.848 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:46:38 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:cf+LtMLOAFrqmfMpz7xFqBbpwDE=', 'Content-Length': '0'} 2023-09-06T18:54:39.848 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.848 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.849 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.849 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.849 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.849 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.849 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.849 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:46:38 GMT 2023-09-06T18:54:39.850 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-6/ 2023-09-06T18:54:39.850 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.850 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:cf+LtMLOAFrqmfMpz7xFqBbpwDE= 2023-09-06T18:54:39.850 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:46:38 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:cf+LtMLOAFrqmfMpz7xFqBbpwDE=', 'Content-Length': '0'} 2023-09-06T18:54:39.850 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.851 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.851 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.851 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.851 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.851 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.852 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.852 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:46:38 GMT 2023-09-06T18:54:39.852 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-6/ 2023-09-06T18:54:39.852 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.852 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:cf+LtMLOAFrqmfMpz7xFqBbpwDE= 2023-09-06T18:54:39.853 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:46:38 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:cf+LtMLOAFrqmfMpz7xFqBbpwDE=', 'Content-Length': '0'} 2023-09-06T18:54:39.853 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.853 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.853 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.853 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.853 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.854 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.854 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.854 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.854 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.855 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1458.143555547 and callback=>; now=1428.143555547; delay=30.0 2023-09-06T18:54:39.855 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.855 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.855 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.855 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.856 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.856 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.856 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.856 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.856 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.857 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.857 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1458.155442447 and callback=>; now=1428.155442447; delay=30.0 2023-09-06T18:54:39.857 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.857 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.857 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.858 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.858 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.858 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.858 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.858 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.859 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:46:38 GMT 2023-09-06T18:54:39.859 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-6/ 2023-09-06T18:54:39.859 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.859 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:cf+LtMLOAFrqmfMpz7xFqBbpwDE= 2023-09-06T18:54:39.859 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:46:38 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:cf+LtMLOAFrqmfMpz7xFqBbpwDE=', 'Content-Length': '0'} 2023-09-06T18:54:39.859 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.860 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.860 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.860 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.860 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.860 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.861 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.861 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:46:38 GMT 2023-09-06T18:54:39.861 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-6/ 2023-09-06T18:54:39.861 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.861 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:cf+LtMLOAFrqmfMpz7xFqBbpwDE= 2023-09-06T18:54:39.862 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:46:38 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:cf+LtMLOAFrqmfMpz7xFqBbpwDE=', 'Content-Length': '0'} 2023-09-06T18:54:39.862 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.862 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.862 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.862 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.863 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.863 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.863 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.863 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:46:38 GMT 2023-09-06T18:54:39.863 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-6/ 2023-09-06T18:54:39.864 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.864 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:cf+LtMLOAFrqmfMpz7xFqBbpwDE= 2023-09-06T18:54:39.864 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:46:38 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:cf+LtMLOAFrqmfMpz7xFqBbpwDE=', 'Content-Length': '0'} 2023-09-06T18:54:39.864 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.864 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.864 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.865 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.865 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.865 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.865 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.865 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:46:38 GMT 2023-09-06T18:54:39.866 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-6/ 2023-09-06T18:54:39.866 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.866 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:cf+LtMLOAFrqmfMpz7xFqBbpwDE= 2023-09-06T18:54:39.866 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:46:38 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:cf+LtMLOAFrqmfMpz7xFqBbpwDE=', 'Content-Length': '0'} 2023-09-06T18:54:39.866 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.867 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.867 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.867 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.867 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.868 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.868 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.868 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.868 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.868 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1488.144423751 and callback=>; now=1458.144423751; delay=30.0 2023-09-06T18:54:39.869 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.869 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.869 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.869 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.869 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.871 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.871 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1488.156613926 and callback=>; now=1458.156613926; delay=30.0 2023-09-06T18:54:39.871 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:--------------------- >> end captured logging << --------------------- 2023-09-06T18:54:39.872 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:====================================================================== 2023-09-06T18:54:39.873 INFO:teuthology.orchestra.run.smithi099.stderr:ERROR: test s3 notification of metadata on master 2023-09-06T18:54:39.873 INFO:teuthology.orchestra.run.smithi099.stderr:---------------------------------------------------------------------- 2023-09-06T18:54:39.873 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.873 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest 2023-09-06T18:54:39.873 INFO:teuthology.orchestra.run.smithi099.stderr: self.test(*self.arg) 2023-09-06T18:54:39.874 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 2555, in test_ps_s3_metadata_on_master 2023-09-06T18:54:39.874 INFO:teuthology.orchestra.run.smithi099.stderr: bucket = conn.create_bucket(bucket_name) 2023-09-06T18:54:39.874 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 619, in create_bucket 2023-09-06T18:54:39.874 INFO:teuthology.orchestra.run.smithi099.stderr: data=data) 2023-09-06T18:54:39.874 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 671, in make_request 2023-09-06T18:54:39.875 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler 2023-09-06T18:54:39.875 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1071, in make_request 2023-09-06T18:54:39.875 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler) 2023-09-06T18:54:39.875 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1030, in _mexe 2023-09-06T18:54:39.875 INFO:teuthology.orchestra.run.smithi099.stderr: raise ex 2023-09-06T18:54:39.875 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 943, in _mexe 2023-09-06T18:54:39.876 INFO:teuthology.orchestra.run.smithi099.stderr: request.body, request.headers) 2023-09-06T18:54:39.876 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.876 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.876 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.876 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.877 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.877 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.877 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1044, in _send_output 2023-09-06T18:54:39.877 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.877 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.878 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.878 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 954, in connect 2023-09-06T18:54:39.878 INFO:teuthology.orchestra.run.smithi099.stderr: (self.host,self.port), self.timeout, self.source_address) 2023-09-06T18:54:39.878 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 724, in create_connection 2023-09-06T18:54:39.878 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.879 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 713, in create_connection 2023-09-06T18:54:39.879 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:-------------------- >> begin captured logging << -------------------- 2023-09-06T18:54:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using access key provided by client. 2023-09-06T18:54:39.880 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using secret key provided by client. 2023-09-06T18:54:39.880 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-7/ 2023-09-06T18:54:39.880 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-7/ 2023-09-06T18:54:39.880 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:39.880 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-7/ 2023-09-06T18:54:39.880 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:39.881 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {} 2023-09-06T18:54:39.881 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:39.881 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:39.881 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:39.881 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.882 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.882 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.882 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.882 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.882 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:47:26 GMT 2023-09-06T18:54:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-7/ 2023-09-06T18:54:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:MmCZk/jcl+ijKwbTw5CfZlCRQZ4= 2023-09-06T18:54:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:47:26 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:MmCZk/jcl+ijKwbTw5CfZlCRQZ4=', 'Content-Length': '0'} 2023-09-06T18:54:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.885 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.885 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.885 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:47:26 GMT 2023-09-06T18:54:39.885 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-7/ 2023-09-06T18:54:39.885 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.885 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:MmCZk/jcl+ijKwbTw5CfZlCRQZ4= 2023-09-06T18:54:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:47:26 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:MmCZk/jcl+ijKwbTw5CfZlCRQZ4=', 'Content-Length': '0'} 2023-09-06T18:54:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.887 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.887 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:47:26 GMT 2023-09-06T18:54:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-7/ 2023-09-06T18:54:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:MmCZk/jcl+ijKwbTw5CfZlCRQZ4= 2023-09-06T18:54:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:47:26 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:MmCZk/jcl+ijKwbTw5CfZlCRQZ4=', 'Content-Length': '0'} 2023-09-06T18:54:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.889 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.890 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.890 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:47:26 GMT 2023-09-06T18:54:39.890 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-7/ 2023-09-06T18:54:39.890 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.890 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:MmCZk/jcl+ijKwbTw5CfZlCRQZ4= 2023-09-06T18:54:39.891 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:47:26 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:MmCZk/jcl+ijKwbTw5CfZlCRQZ4=', 'Content-Length': '0'} 2023-09-06T18:54:39.891 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.891 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.891 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.891 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.892 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.892 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.892 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.892 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:47:26 GMT 2023-09-06T18:54:39.892 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-7/ 2023-09-06T18:54:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:MmCZk/jcl+ijKwbTw5CfZlCRQZ4= 2023-09-06T18:54:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:47:26 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:MmCZk/jcl+ijKwbTw5CfZlCRQZ4=', 'Content-Length': '0'} 2023-09-06T18:54:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.894 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.894 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.894 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.894 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1518.14570956 and callback=>; now=1488.14570956; delay=30.0 2023-09-06T18:54:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.896 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.896 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.896 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.896 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.896 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.897 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.897 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.897 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.897 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.897 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1518.157385527 and callback=>; now=1488.157385527; delay=30.0 2023-09-06T18:54:39.897 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.898 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.898 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.898 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 13 heartbeat frames, sent 13, idle intervals 0 2023-09-06T18:54:39.898 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1553.162329771 and callback=>; now=1488.162329771; delay=65 2023-09-06T18:54:39.898 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 13 heartbeat frames, sent 13, idle intervals 0 2023-09-06T18:54:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1553.172924974 and callback=>; now=1488.172924974; delay=65 2023-09-06T18:54:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.899 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.900 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:47:26 GMT 2023-09-06T18:54:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-7/ 2023-09-06T18:54:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:MmCZk/jcl+ijKwbTw5CfZlCRQZ4= 2023-09-06T18:54:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:47:26 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:MmCZk/jcl+ijKwbTw5CfZlCRQZ4=', 'Content-Length': '0'} 2023-09-06T18:54:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1548.146493711 and callback=>; now=1518.146493711; delay=30.0 2023-09-06T18:54:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1548.158551773 and callback=>; now=1518.158551773; delay=30.0 2023-09-06T18:54:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.906 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.906 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:47:26 GMT 2023-09-06T18:54:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-7/ 2023-09-06T18:54:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:MmCZk/jcl+ijKwbTw5CfZlCRQZ4= 2023-09-06T18:54:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:47:26 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:MmCZk/jcl+ijKwbTw5CfZlCRQZ4=', 'Content-Length': '0'} 2023-09-06T18:54:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1578.147764896 and callback=>; now=1548.147764896; delay=30.0 2023-09-06T18:54:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.911 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.911 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.911 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.911 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.911 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.912 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1578.159511282 and callback=>; now=1548.159511282; delay=30.0 2023-09-06T18:54:39.912 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.912 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.912 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.912 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 15 heartbeat frames, sent 15, idle intervals 0 2023-09-06T18:54:39.913 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1618.163329433 and callback=>; now=1553.163329433; delay=65 2023-09-06T18:54:39.913 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 15 heartbeat frames, sent 15, idle intervals 0 2023-09-06T18:54:39.913 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1618.174042791 and callback=>; now=1553.174042791; delay=65 2023-09-06T18:54:39.913 INFO:teuthology.orchestra.run.smithi099.stderr:--------------------- >> end captured logging << --------------------- 2023-09-06T18:54:39.913 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.914 INFO:teuthology.orchestra.run.smithi099.stderr:====================================================================== 2023-09-06T18:54:39.914 INFO:teuthology.orchestra.run.smithi099.stderr:ERROR: test s3 notification of tags on master 2023-09-06T18:54:39.914 INFO:teuthology.orchestra.run.smithi099.stderr:---------------------------------------------------------------------- 2023-09-06T18:54:39.914 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.914 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest 2023-09-06T18:54:39.915 INFO:teuthology.orchestra.run.smithi099.stderr: self.test(*self.arg) 2023-09-06T18:54:39.915 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 2649, in test_ps_s3_tags_on_master 2023-09-06T18:54:39.915 INFO:teuthology.orchestra.run.smithi099.stderr: bucket = conn.create_bucket(bucket_name) 2023-09-06T18:54:39.915 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 619, in create_bucket 2023-09-06T18:54:39.915 INFO:teuthology.orchestra.run.smithi099.stderr: data=data) 2023-09-06T18:54:39.915 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 671, in make_request 2023-09-06T18:54:39.916 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler 2023-09-06T18:54:39.916 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1071, in make_request 2023-09-06T18:54:39.916 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler) 2023-09-06T18:54:39.916 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1030, in _mexe 2023-09-06T18:54:39.916 INFO:teuthology.orchestra.run.smithi099.stderr: raise ex 2023-09-06T18:54:39.917 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 943, in _mexe 2023-09-06T18:54:39.917 INFO:teuthology.orchestra.run.smithi099.stderr: request.body, request.headers) 2023-09-06T18:54:39.917 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.917 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.917 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.918 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.918 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.918 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.918 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1044, in _send_output 2023-09-06T18:54:39.918 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.919 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.919 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.919 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 954, in connect 2023-09-06T18:54:39.919 INFO:teuthology.orchestra.run.smithi099.stderr: (self.host,self.port), self.timeout, self.source_address) 2023-09-06T18:54:39.919 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 724, in create_connection 2023-09-06T18:54:39.920 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.920 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 713, in create_connection 2023-09-06T18:54:39.920 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.920 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.920 INFO:teuthology.orchestra.run.smithi099.stderr:-------------------- >> begin captured logging << -------------------- 2023-09-06T18:54:39.921 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using access key provided by client. 2023-09-06T18:54:39.921 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using secret key provided by client. 2023-09-06T18:54:39.921 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-8/ 2023-09-06T18:54:39.921 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-8/ 2023-09-06T18:54:39.921 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:39.922 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-8/ 2023-09-06T18:54:39.922 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:39.922 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {} 2023-09-06T18:54:39.922 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:39.922 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:39.923 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:39.923 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.923 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.923 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.923 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.924 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.924 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.924 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:48:49 GMT 2023-09-06T18:54:39.924 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-8/ 2023-09-06T18:54:39.924 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.924 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:BHYHQ4Cm7RncksEUq1OWH4kIfNI= 2023-09-06T18:54:39.925 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:48:49 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:BHYHQ4Cm7RncksEUq1OWH4kIfNI=', 'Content-Length': '0'} 2023-09-06T18:54:39.925 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.925 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.925 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.925 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.926 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.926 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.926 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.926 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:48:49 GMT 2023-09-06T18:54:39.927 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-8/ 2023-09-06T18:54:39.927 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.927 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:BHYHQ4Cm7RncksEUq1OWH4kIfNI= 2023-09-06T18:54:39.927 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:48:49 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:BHYHQ4Cm7RncksEUq1OWH4kIfNI=', 'Content-Length': '0'} 2023-09-06T18:54:39.927 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.928 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.928 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.928 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.928 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.928 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.929 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.929 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:48:49 GMT 2023-09-06T18:54:39.929 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-8/ 2023-09-06T18:54:39.929 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.929 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:BHYHQ4Cm7RncksEUq1OWH4kIfNI= 2023-09-06T18:54:39.929 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:48:49 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:BHYHQ4Cm7RncksEUq1OWH4kIfNI=', 'Content-Length': '0'} 2023-09-06T18:54:39.930 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.930 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.930 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.930 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.930 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.931 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.931 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.931 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:48:49 GMT 2023-09-06T18:54:39.931 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-8/ 2023-09-06T18:54:39.931 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.932 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:BHYHQ4Cm7RncksEUq1OWH4kIfNI= 2023-09-06T18:54:39.932 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:48:49 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:BHYHQ4Cm7RncksEUq1OWH4kIfNI=', 'Content-Length': '0'} 2023-09-06T18:54:39.932 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.932 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.932 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.933 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.933 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:48:49 GMT 2023-09-06T18:54:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-8/ 2023-09-06T18:54:39.934 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.934 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:BHYHQ4Cm7RncksEUq1OWH4kIfNI= 2023-09-06T18:54:39.934 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:48:49 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:BHYHQ4Cm7RncksEUq1OWH4kIfNI=', 'Content-Length': '0'} 2023-09-06T18:54:39.934 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.934 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.935 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.935 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.935 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.935 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.935 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.936 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:48:49 GMT 2023-09-06T18:54:39.936 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-8/ 2023-09-06T18:54:39.936 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.936 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:BHYHQ4Cm7RncksEUq1OWH4kIfNI= 2023-09-06T18:54:39.937 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:48:49 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:BHYHQ4Cm7RncksEUq1OWH4kIfNI=', 'Content-Length': '0'} 2023-09-06T18:54:39.937 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.937 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.937 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.937 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.937 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.938 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.938 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.938 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1608.148972573 and callback=>; now=1578.148972573; delay=30.0 2023-09-06T18:54:39.938 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.938 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.939 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.939 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.939 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.939 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.939 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.939 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.940 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.940 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.940 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.940 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.940 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1608.160527539 and callback=>; now=1578.160527539; delay=30.0 2023-09-06T18:54:39.941 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.941 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.941 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.941 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.941 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.941 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.942 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.942 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.942 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:48:49 GMT 2023-09-06T18:54:39.942 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-8/ 2023-09-06T18:54:39.942 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.943 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:BHYHQ4Cm7RncksEUq1OWH4kIfNI= 2023-09-06T18:54:39.943 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:48:49 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:BHYHQ4Cm7RncksEUq1OWH4kIfNI=', 'Content-Length': '0'} 2023-09-06T18:54:39.943 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.943 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.943 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.944 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.944 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.944 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.945 INFO:teuthology.orchestra.run.smithi099.stdout:send: b'GET /cjhpse-1?notification=cjhpse-1_notif_4 HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nAuthorization: AWS DPCRSOKICJWGAVMTJRSS:V4L5V5Q/CFWNKOT23E7IRZnaaG0=\r\nDate: Wed, 06 Sep 2023 18:41:01 +0000\r\nHost: smithi099.front.sepia.ceph.com:80\r\n\r\n' 2023-09-06T18:54:39.945 INFO:teuthology.orchestra.run.smithi099.stdout:Zero length chunk ignored 2023-09-06T18:54:39.945 INFO:teuthology.orchestra.run.smithi099.stdout:reply: 'HTTP/1.1 200 OK\r\n' 2023-09-06T18:54:39.945 INFO:teuthology.orchestra.run.smithi099.stdout:header: x-amz-request-id: tx000008a26116e69eeeeb0-0064f8c7bd-11b2-default 2023-09-06T18:54:39.946 INFO:teuthology.orchestra.run.smithi099.stdout:header: Content-Type: application/xml 2023-09-06T18:54:39.946 INFO:teuthology.orchestra.run.smithi099.stdout:header: Server: Ceph Object Gateway (reef) 2023-09-06T18:54:39.946 INFO:teuthology.orchestra.run.smithi099.stdout:header: Content-Length: 504 2023-09-06T18:54:39.946 INFO:teuthology.orchestra.run.smithi099.stdout:header: Date: Wed, 06 Sep 2023 18:41:01 GMT 2023-09-06T18:54:39.946 INFO:teuthology.orchestra.run.smithi099.stdout:header: Connection: Keep-Alive 2023-09-06T18:54:39.947 INFO:teuthology.orchestra.run.smithi099.stdout:wait for 5sec for the messages... 2023-09-06T18:54:39.947 INFO:teuthology.orchestra.run.smithi099.stdout:send: b'DELETE /cjhpse-1?notification HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nAuthorization: AWS DPCRSOKICJWGAVMTJRSS:j2gUjr4Hj66g3VmW+L4VbaCiNao=\r\nDate: Wed, 06 Sep 2023 18:41:10 +0000\r\nHost: smithi099.front.sepia.ceph.com:80\r\n\r\n' 2023-09-06T18:54:39.947 INFO:teuthology.orchestra.run.smithi099.stdout:Zero length chunk ignored 2023-09-06T18:54:39.947 INFO:teuthology.orchestra.run.smithi099.stdout:reply: 'HTTP/1.1 200 OK\r\n' 2023-09-06T18:54:39.947 INFO:teuthology.orchestra.run.smithi099.stdout:header: x-amz-request-id: tx00000e48561df4fa723ff-0064f8c7c6-11b2-default 2023-09-06T18:54:39.947 INFO:teuthology.orchestra.run.smithi099.stdout:header: Server: Ceph Object Gateway (reef) 2023-09-06T18:54:39.948 INFO:teuthology.orchestra.run.smithi099.stdout:header: Content-Length: 0 2023-09-06T18:54:39.948 INFO:teuthology.orchestra.run.smithi099.stdout:header: Date: Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:39.948 INFO:teuthology.orchestra.run.smithi099.stdout:header: Connection: Keep-Alive 2023-09-06T18:54:39.948 INFO:teuthology.orchestra.run.smithi099.stdout:send: b'DELETE /cjhpse-1?notification HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nAuthorization: AWS DPCRSOKICJWGAVMTJRSS:j2gUjr4Hj66g3VmW+L4VbaCiNao=\r\nDate: Wed, 06 Sep 2023 18:41:10 +0000\r\nHost: smithi099.front.sepia.ceph.com:80\r\n\r\n' 2023-09-06T18:54:39.948 INFO:teuthology.orchestra.run.smithi099.stdout:Zero length chunk ignored 2023-09-06T18:54:39.949 INFO:teuthology.orchestra.run.smithi099.stdout:reply: 'HTTP/1.1 200 OK\r\n' 2023-09-06T18:54:39.949 INFO:teuthology.orchestra.run.smithi099.stdout:header: x-amz-request-id: tx0000094e39f7a846a2d2c-0064f8c7c6-11b2-default 2023-09-06T18:54:39.949 INFO:teuthology.orchestra.run.smithi099.stdout:header: Server: Ceph Object Gateway (reef) 2023-09-06T18:54:39.949 INFO:teuthology.orchestra.run.smithi099.stdout:header: Content-Length: 0 2023-09-06T18:54:39.949 INFO:teuthology.orchestra.run.smithi099.stdout:header: Date: Wed, 06 Sep 2023 18:41:10 GMT 2023-09-06T18:54:39.950 INFO:teuthology.orchestra.run.smithi099.stdout:header: Connection: Keep-Alive 2023-09-06T18:54:39.950 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.950 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.950 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.950 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.951 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.951 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.951 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.951 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.951 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.951 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.952 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.952 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.952 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.952 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.952 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.953 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.953 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.953 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.953 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.953 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.953 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.954 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.954 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.954 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.954 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.954 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.955 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.955 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.955 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.955 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.955 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.955 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.956 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.956 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.956 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.956 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.956 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.956 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.957 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.957 INFO:teuthology.orchestra.run.smithi099.stdout:Error: [Errno 111] Connection refused 2023-09-06T18:54:39.957 INFO:teuthology.orchestra.run.smithi099.stdout:average time for creation + qmqp notification is: 1115.5087852478027 milliseconds 2023-09-06T18:54:39.957 INFO:teuthology.orchestra.run.smithi099.stdout:wait for 5sec for the messages... 2023-09-06T18:54:39.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-06T18:54:39.959 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.959 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.959 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.960 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1638.149628167 and callback=>; now=1608.149628167; delay=30.0 2023-09-06T18:54:39.960 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.960 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.960 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.961 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.961 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.961 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.961 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.961 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.961 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.962 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.962 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1638.161483381 and callback=>; now=1608.161483381; delay=30.0 2023-09-06T18:54:39.962 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.962 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.962 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.963 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 17 heartbeat frames, sent 17, idle intervals 0 2023-09-06T18:54:39.963 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1683.164498468 and callback=>; now=1618.164498468; delay=65 2023-09-06T18:54:39.963 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 17 heartbeat frames, sent 17, idle intervals 0 2023-09-06T18:54:39.963 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1683.175163407 and callback=>; now=1618.175163407; delay=65 2023-09-06T18:54:39.963 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.963 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.965 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1668.150623112 and callback=>; now=1638.150623112; delay=30.0 2023-09-06T18:54:39.965 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.965 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.965 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.965 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.966 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.966 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.966 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.966 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.966 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.966 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.967 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1668.16246836 and callback=>; now=1638.16246836; delay=30.0 2023-09-06T18:54:39.967 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.967 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.967 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:--------------------- >> end captured logging << --------------------- 2023-09-06T18:54:39.968 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:====================================================================== 2023-09-06T18:54:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:ERROR: test s3 notification of object versions 2023-09-06T18:54:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:---------------------------------------------------------------------- 2023-09-06T18:54:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:39.969 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest 2023-09-06T18:54:39.969 INFO:teuthology.orchestra.run.smithi099.stderr: self.test(*self.arg) 2023-09-06T18:54:39.969 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 2760, in test_ps_s3_versioning_on_master 2023-09-06T18:54:39.969 INFO:teuthology.orchestra.run.smithi099.stderr: bucket = conn.create_bucket(bucket_name) 2023-09-06T18:54:39.969 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 619, in create_bucket 2023-09-06T18:54:39.969 INFO:teuthology.orchestra.run.smithi099.stderr: data=data) 2023-09-06T18:54:39.970 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 671, in make_request 2023-09-06T18:54:39.970 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler 2023-09-06T18:54:39.970 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1071, in make_request 2023-09-06T18:54:39.970 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler) 2023-09-06T18:54:39.970 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1030, in _mexe 2023-09-06T18:54:39.971 INFO:teuthology.orchestra.run.smithi099.stderr: raise ex 2023-09-06T18:54:39.971 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 943, in _mexe 2023-09-06T18:54:39.971 INFO:teuthology.orchestra.run.smithi099.stderr: request.body, request.headers) 2023-09-06T18:54:39.971 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:39.971 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:39.972 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:39.972 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.972 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:39.972 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:39.972 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1044, in _send_output 2023-09-06T18:54:39.972 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:39.973 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:39.973 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:39.973 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 954, in connect 2023-09-06T18:54:39.973 INFO:teuthology.orchestra.run.smithi099.stderr: (self.host,self.port), self.timeout, self.source_address) 2023-09-06T18:54:39.973 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 724, in create_connection 2023-09-06T18:54:39.973 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:39.974 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 713, in create_connection 2023-09-06T18:54:39.974 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:39.974 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:39.974 INFO:teuthology.orchestra.run.smithi099.stderr:-------------------- >> begin captured logging << -------------------- 2023-09-06T18:54:39.974 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using access key provided by client. 2023-09-06T18:54:39.975 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using secret key provided by client. 2023-09-06T18:54:39.975 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-9/ 2023-09-06T18:54:39.975 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-9/ 2023-09-06T18:54:39.975 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:39.975 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-9/ 2023-09-06T18:54:39.975 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:39.976 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {} 2023-09-06T18:54:39.976 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:39.976 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:39.976 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:39.976 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.977 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.977 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.977 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.977 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.977 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.977 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:50:11 GMT 2023-09-06T18:54:39.978 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-9/ 2023-09-06T18:54:39.978 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.978 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:QYDTdzqR8YJgqsDfERIpHFYl2lE= 2023-09-06T18:54:39.978 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:50:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:QYDTdzqR8YJgqsDfERIpHFYl2lE=', 'Content-Length': '0'} 2023-09-06T18:54:39.978 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.979 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.979 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.979 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.979 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.979 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.979 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.980 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:50:11 GMT 2023-09-06T18:54:39.980 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-9/ 2023-09-06T18:54:39.980 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.980 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:QYDTdzqR8YJgqsDfERIpHFYl2lE= 2023-09-06T18:54:39.981 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:50:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:QYDTdzqR8YJgqsDfERIpHFYl2lE=', 'Content-Length': '0'} 2023-09-06T18:54:39.981 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.981 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.981 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.981 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.982 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.982 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.982 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.982 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:50:11 GMT 2023-09-06T18:54:39.982 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-9/ 2023-09-06T18:54:39.983 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.983 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:QYDTdzqR8YJgqsDfERIpHFYl2lE= 2023-09-06T18:54:39.983 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:50:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:QYDTdzqR8YJgqsDfERIpHFYl2lE=', 'Content-Length': '0'} 2023-09-06T18:54:39.983 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.983 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.984 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.984 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.984 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.984 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.984 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.985 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:50:11 GMT 2023-09-06T18:54:39.985 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-9/ 2023-09-06T18:54:39.985 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.985 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:QYDTdzqR8YJgqsDfERIpHFYl2lE= 2023-09-06T18:54:39.985 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:50:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:QYDTdzqR8YJgqsDfERIpHFYl2lE=', 'Content-Length': '0'} 2023-09-06T18:54:39.985 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.986 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.986 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.986 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.986 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.986 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.987 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.989 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:50:11 GMT 2023-09-06T18:54:39.989 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-9/ 2023-09-06T18:54:39.989 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.989 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:QYDTdzqR8YJgqsDfERIpHFYl2lE= 2023-09-06T18:54:39.990 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:50:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:QYDTdzqR8YJgqsDfERIpHFYl2lE=', 'Content-Length': '0'} 2023-09-06T18:54:39.990 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.990 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.990 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.991 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.991 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.991 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.991 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.991 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:50:11 GMT 2023-09-06T18:54:39.992 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-9/ 2023-09-06T18:54:39.992 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.992 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:QYDTdzqR8YJgqsDfERIpHFYl2lE= 2023-09-06T18:54:39.992 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:50:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:QYDTdzqR8YJgqsDfERIpHFYl2lE=', 'Content-Length': '0'} 2023-09-06T18:54:39.992 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.992 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.993 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.993 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.993 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.993 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.993 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:39.994 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:39.994 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.994 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1698.151633869 and callback=>; now=1668.151633869; delay=30.0 2023-09-06T18:54:39.994 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:39.994 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:39.995 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.995 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:39.995 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:39.995 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.995 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:39.996 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:39.996 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:39.996 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:39.996 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1698.163463023 and callback=>; now=1668.163463023; delay=30.0 2023-09-06T18:54:39.996 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:39.996 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:39.997 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:39.997 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:39.997 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:39.997 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:39.997 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.997 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:39.998 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:50:11 GMT 2023-09-06T18:54:39.998 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-9/ 2023-09-06T18:54:39.998 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:39.998 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:QYDTdzqR8YJgqsDfERIpHFYl2lE= 2023-09-06T18:54:39.999 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:50:11 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:QYDTdzqR8YJgqsDfERIpHFYl2lE=', 'Content-Length': '0'} 2023-09-06T18:54:39.999 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:39.999 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:39.999 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 19 heartbeat frames, sent 19, idle intervals 0 2023-09-06T18:54:39.999 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1748.167438731 and callback=>; now=1683.167438731; delay=65 2023-09-06T18:54:40.000 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 19 heartbeat frames, sent 19, idle intervals 0 2023-09-06T18:54:40.000 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1748.178897063 and callback=>; now=1683.178897063; delay=65 2023-09-06T18:54:40.000 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:40.000 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:40.000 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.001 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.001 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:40.001 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:40.001 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:40.001 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1728.152460645 and callback=>; now=1698.152460645; delay=30.0 2023-09-06T18:54:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:40.003 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.003 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.003 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:40.003 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:40.003 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:40.004 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1728.164465094 and callback=>; now=1698.164465094; delay=30.0 2023-09-06T18:54:40.004 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:40.004 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:40.004 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:40.004 INFO:teuthology.orchestra.run.smithi099.stderr:--------------------- >> end captured logging << --------------------- 2023-09-06T18:54:40.005 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.005 INFO:teuthology.orchestra.run.smithi099.stderr:====================================================================== 2023-09-06T18:54:40.005 INFO:teuthology.orchestra.run.smithi099.stderr:ERROR: test s3 notification of deletion markers on master 2023-09-06T18:54:40.005 INFO:teuthology.orchestra.run.smithi099.stderr:---------------------------------------------------------------------- 2023-09-06T18:54:40.005 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:40.005 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest 2023-09-06T18:54:40.006 INFO:teuthology.orchestra.run.smithi099.stderr: self.test(*self.arg) 2023-09-06T18:54:40.006 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 2835, in test_ps_s3_versioned_deletion_on_master 2023-09-06T18:54:40.006 INFO:teuthology.orchestra.run.smithi099.stderr: bucket = conn.create_bucket(bucket_name) 2023-09-06T18:54:40.006 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 619, in create_bucket 2023-09-06T18:54:40.007 INFO:teuthology.orchestra.run.smithi099.stderr: data=data) 2023-09-06T18:54:40.007 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 671, in make_request 2023-09-06T18:54:40.007 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler 2023-09-06T18:54:40.007 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1071, in make_request 2023-09-06T18:54:40.007 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler) 2023-09-06T18:54:40.008 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1030, in _mexe 2023-09-06T18:54:40.008 INFO:teuthology.orchestra.run.smithi099.stderr: raise ex 2023-09-06T18:54:40.008 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 943, in _mexe 2023-09-06T18:54:40.008 INFO:teuthology.orchestra.run.smithi099.stderr: request.body, request.headers) 2023-09-06T18:54:40.008 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:40.008 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:40.009 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:40.009 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:40.009 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:40.009 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:40.009 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1044, in _send_output 2023-09-06T18:54:40.010 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:40.010 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:40.010 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:40.010 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 954, in connect 2023-09-06T18:54:40.010 INFO:teuthology.orchestra.run.smithi099.stderr: (self.host,self.port), self.timeout, self.source_address) 2023-09-06T18:54:40.010 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 724, in create_connection 2023-09-06T18:54:40.011 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:40.011 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 713, in create_connection 2023-09-06T18:54:40.011 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:-------------------- >> begin captured logging << -------------------- 2023-09-06T18:54:40.012 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using access key provided by client. 2023-09-06T18:54:40.012 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using secret key provided by client. 2023-09-06T18:54:40.012 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-10/ 2023-09-06T18:54:40.012 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-10/ 2023-09-06T18:54:40.012 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:40.013 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-10/ 2023-09-06T18:54:40.013 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:40.013 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {} 2023-09-06T18:54:40.013 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:40.013 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:40.013 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:40.014 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.014 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.014 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.014 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.014 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.015 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:51:30 GMT 2023-09-06T18:54:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-10/ 2023-09-06T18:54:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:8VxziJrgT26vg78rf1NuDKJhZTo= 2023-09-06T18:54:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:51:30 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:8VxziJrgT26vg78rf1NuDKJhZTo=', 'Content-Length': '0'} 2023-09-06T18:54:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.016 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.017 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.017 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:51:30 GMT 2023-09-06T18:54:40.017 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-10/ 2023-09-06T18:54:40.017 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.017 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:8VxziJrgT26vg78rf1NuDKJhZTo= 2023-09-06T18:54:40.017 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:51:30 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:8VxziJrgT26vg78rf1NuDKJhZTo=', 'Content-Length': '0'} 2023-09-06T18:54:40.018 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.018 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.018 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.018 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.018 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.019 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.019 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.019 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:51:30 GMT 2023-09-06T18:54:40.019 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-10/ 2023-09-06T18:54:40.019 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.019 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:8VxziJrgT26vg78rf1NuDKJhZTo= 2023-09-06T18:54:40.020 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:51:30 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:8VxziJrgT26vg78rf1NuDKJhZTo=', 'Content-Length': '0'} 2023-09-06T18:54:40.020 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.020 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.020 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.020 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.021 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.021 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.021 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.021 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:51:30 GMT 2023-09-06T18:54:40.021 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-10/ 2023-09-06T18:54:40.021 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.022 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:8VxziJrgT26vg78rf1NuDKJhZTo= 2023-09-06T18:54:40.022 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:51:30 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:8VxziJrgT26vg78rf1NuDKJhZTo=', 'Content-Length': '0'} 2023-09-06T18:54:40.022 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.022 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.022 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.023 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.023 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.023 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.023 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.023 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:51:30 GMT 2023-09-06T18:54:40.024 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-10/ 2023-09-06T18:54:40.024 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.024 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:8VxziJrgT26vg78rf1NuDKJhZTo= 2023-09-06T18:54:40.024 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:51:30 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:8VxziJrgT26vg78rf1NuDKJhZTo=', 'Content-Length': '0'} 2023-09-06T18:54:40.025 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.025 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.025 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:40.025 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:40.025 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.026 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.026 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:40.026 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:40.026 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:40.026 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1758.153596408 and callback=>; now=1728.153596408; delay=30.0 2023-09-06T18:54:40.027 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:40.027 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:40.027 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:40.027 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:40.028 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:40.028 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.028 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.029 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:40.029 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:40.029 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:40.030 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1758.165609501 and callback=>; now=1728.165609501; delay=30.0 2023-09-06T18:54:40.030 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:40.030 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:40.030 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:40.031 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.031 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.031 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.032 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.032 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.032 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:51:30 GMT 2023-09-06T18:54:40.032 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-10/ 2023-09-06T18:54:40.032 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.033 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:8VxziJrgT26vg78rf1NuDKJhZTo= 2023-09-06T18:54:40.033 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:51:30 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:8VxziJrgT26vg78rf1NuDKJhZTo=', 'Content-Length': '0'} 2023-09-06T18:54:40.033 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.033 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.033 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 21 heartbeat frames, sent 21, idle intervals 0 2023-09-06T18:54:40.034 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1813.17225143 and callback=>; now=1748.17225143; delay=65 2023-09-06T18:54:40.034 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 21 heartbeat frames, sent 21, idle intervals 0 2023-09-06T18:54:40.034 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1813.184104286 and callback=>; now=1748.184104286; delay=65 2023-09-06T18:54:40.034 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:40.034 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:40.035 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.035 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.035 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:40.035 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:40.035 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:40.036 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1788.154895265 and callback=>; now=1758.154895265; delay=30.0 2023-09-06T18:54:40.036 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:40.036 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:40.036 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:40.036 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:40.036 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:40.037 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.037 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.037 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:40.037 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:40.038 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:40.038 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1788.166652376 and callback=>; now=1758.166652376; delay=30.0 2023-09-06T18:54:40.038 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:40.038 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:40.038 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:40.039 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.039 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.039 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.039 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.039 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.040 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:51:30 GMT 2023-09-06T18:54:40.040 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-10/ 2023-09-06T18:54:40.040 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.040 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:8VxziJrgT26vg78rf1NuDKJhZTo= 2023-09-06T18:54:40.040 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:51:30 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:8VxziJrgT26vg78rf1NuDKJhZTo=', 'Content-Length': '0'} 2023-09-06T18:54:40.041 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.041 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.041 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:40.041 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:40.041 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.041 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.042 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:40.042 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:40.042 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:40.042 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1818.155631336 and callback=>; now=1788.155631336; delay=30.0 2023-09-06T18:54:40.043 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:40.043 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:40.043 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:40.043 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:40.043 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:40.044 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.044 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.044 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:40.044 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:40.044 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1818.167354729 and callback=>; now=1788.167354729; delay=30.0 2023-09-06T18:54:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:--------------------- >> end captured logging << --------------------- 2023-09-06T18:54:40.046 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:====================================================================== 2023-09-06T18:54:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:ERROR: test pushing persistent notification amqp 2023-09-06T18:54:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:---------------------------------------------------------------------- 2023-09-06T18:54:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:40.047 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest 2023-09-06T18:54:40.047 INFO:teuthology.orchestra.run.smithi099.stderr: self.test(*self.arg) 2023-09-06T18:54:40.047 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 3748, in test_ps_s3_persistent_notification_amqp 2023-09-06T18:54:40.047 INFO:teuthology.orchestra.run.smithi099.stderr: persistent_notification('amqp') 2023-09-06T18:54:40.047 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 3639, in persistent_notification 2023-09-06T18:54:40.047 INFO:teuthology.orchestra.run.smithi099.stderr: bucket = conn.create_bucket(bucket_name) 2023-09-06T18:54:40.048 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 619, in create_bucket 2023-09-06T18:54:40.048 INFO:teuthology.orchestra.run.smithi099.stderr: data=data) 2023-09-06T18:54:40.048 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 671, in make_request 2023-09-06T18:54:40.048 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler 2023-09-06T18:54:40.048 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1071, in make_request 2023-09-06T18:54:40.049 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler) 2023-09-06T18:54:40.049 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1030, in _mexe 2023-09-06T18:54:40.049 INFO:teuthology.orchestra.run.smithi099.stderr: raise ex 2023-09-06T18:54:40.049 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 943, in _mexe 2023-09-06T18:54:40.049 INFO:teuthology.orchestra.run.smithi099.stderr: request.body, request.headers) 2023-09-06T18:54:40.050 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:40.050 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:40.050 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:40.050 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:40.050 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:40.051 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:40.051 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1044, in _send_output 2023-09-06T18:54:40.051 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:40.051 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:40.052 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:40.052 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 954, in connect 2023-09-06T18:54:40.052 INFO:teuthology.orchestra.run.smithi099.stderr: (self.host,self.port), self.timeout, self.source_address) 2023-09-06T18:54:40.052 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 724, in create_connection 2023-09-06T18:54:40.052 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:40.053 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 713, in create_connection 2023-09-06T18:54:40.053 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:40.053 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:40.053 INFO:teuthology.orchestra.run.smithi099.stderr:-------------------- >> begin captured logging << -------------------- 2023-09-06T18:54:40.053 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using access key provided by client. 2023-09-06T18:54:40.053 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using secret key provided by client. 2023-09-06T18:54:40.054 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-11/ 2023-09-06T18:54:40.054 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-11/ 2023-09-06T18:54:40.054 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:40.054 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-11/ 2023-09-06T18:54:40.054 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:40.055 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {} 2023-09-06T18:54:40.055 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:40.055 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:40.055 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:40.055 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.056 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.056 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.056 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.056 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.056 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.056 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:52:52 GMT 2023-09-06T18:54:40.057 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-11/ 2023-09-06T18:54:40.057 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.057 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:bVa0nGYJPxIeg9NAfRaLj6uwinw= 2023-09-06T18:54:40.057 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:52:52 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:bVa0nGYJPxIeg9NAfRaLj6uwinw=', 'Content-Length': '0'} 2023-09-06T18:54:40.057 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.058 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.058 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.058 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.058 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.059 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.059 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.059 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:52:52 GMT 2023-09-06T18:54:40.059 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-11/ 2023-09-06T18:54:40.059 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.059 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:bVa0nGYJPxIeg9NAfRaLj6uwinw= 2023-09-06T18:54:40.060 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:52:52 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:bVa0nGYJPxIeg9NAfRaLj6uwinw=', 'Content-Length': '0'} 2023-09-06T18:54:40.060 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.060 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.060 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.060 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.061 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.061 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.061 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.061 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:52:52 GMT 2023-09-06T18:54:40.061 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-11/ 2023-09-06T18:54:40.062 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.062 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:bVa0nGYJPxIeg9NAfRaLj6uwinw= 2023-09-06T18:54:40.062 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:52:52 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:bVa0nGYJPxIeg9NAfRaLj6uwinw=', 'Content-Length': '0'} 2023-09-06T18:54:40.062 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.062 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.063 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.063 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.063 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.063 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.063 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.064 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:52:52 GMT 2023-09-06T18:54:40.064 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-11/ 2023-09-06T18:54:40.064 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.064 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:bVa0nGYJPxIeg9NAfRaLj6uwinw= 2023-09-06T18:54:40.064 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:52:52 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:bVa0nGYJPxIeg9NAfRaLj6uwinw=', 'Content-Length': '0'} 2023-09-06T18:54:40.065 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.065 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.065 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.065 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.065 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.065 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.066 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.066 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:52:52 GMT 2023-09-06T18:54:40.066 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-11/ 2023-09-06T18:54:40.066 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:bVa0nGYJPxIeg9NAfRaLj6uwinw= 2023-09-06T18:54:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:52:52 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:bVa0nGYJPxIeg9NAfRaLj6uwinw=', 'Content-Length': '0'} 2023-09-06T18:54:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 23 heartbeat frames, sent 23, idle intervals 0 2023-09-06T18:54:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1878.177616089 and callback=>; now=1813.177616089; delay=65 2023-09-06T18:54:40.068 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 23 heartbeat frames, sent 23, idle intervals 0 2023-09-06T18:54:40.068 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1878.189453938 and callback=>; now=1813.189453938; delay=65 2023-09-06T18:54:40.068 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:40.068 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:40.069 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.069 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.069 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:40.069 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:40.069 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:40.070 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1848.156529902 and callback=>; now=1818.156529902; delay=30.0 2023-09-06T18:54:40.070 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:40.070 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:40.070 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:40.070 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1848.168359028 and callback=>; now=1818.168359028; delay=30.0 2023-09-06T18:54:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:40.073 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.073 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.073 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.073 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.073 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.073 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:52:52 GMT 2023-09-06T18:54:40.074 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-11/ 2023-09-06T18:54:40.074 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.074 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:bVa0nGYJPxIeg9NAfRaLj6uwinw= 2023-09-06T18:54:40.074 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:52:52 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:bVa0nGYJPxIeg9NAfRaLj6uwinw=', 'Content-Length': '0'} 2023-09-06T18:54:40.075 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.075 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.075 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.075 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.075 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.076 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.076 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.076 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:52:52 GMT 2023-09-06T18:54:40.076 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-11/ 2023-09-06T18:54:40.076 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.076 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:bVa0nGYJPxIeg9NAfRaLj6uwinw= 2023-09-06T18:54:40.077 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:52:52 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:bVa0nGYJPxIeg9NAfRaLj6uwinw=', 'Content-Length': '0'} 2023-09-06T18:54:40.077 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.077 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.077 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:40.077 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:40.078 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.078 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.078 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:40.078 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:40.078 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:40.079 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1878.157710889 and callback=>; now=1848.157710889; delay=30.0 2023-09-06T18:54:40.079 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:40.079 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:40.079 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:40.079 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:40.080 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:40.080 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.080 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.080 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:40.080 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:40.081 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:40.081 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1878.169479386 and callback=>; now=1848.169479386; delay=30.0 2023-09-06T18:54:40.081 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:40.081 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:40.081 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:40.082 INFO:teuthology.orchestra.run.smithi099.stderr:--------------------- >> end captured logging << --------------------- 2023-09-06T18:54:40.082 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.082 INFO:teuthology.orchestra.run.smithi099.stderr:====================================================================== 2023-09-06T18:54:40.082 INFO:teuthology.orchestra.run.smithi099.stderr:ERROR: test pushing persistent notification of large notifications 2023-09-06T18:54:40.083 INFO:teuthology.orchestra.run.smithi099.stderr:---------------------------------------------------------------------- 2023-09-06T18:54:40.083 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2023-09-06T18:54:40.083 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest 2023-09-06T18:54:40.083 INFO:teuthology.orchestra.run.smithi099.stderr: self.test(*self.arg) 2023-09-06T18:54:40.083 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 3772, in test_ps_s3_persistent_notification_large 2023-09-06T18:54:40.083 INFO:teuthology.orchestra.run.smithi099.stderr: bucket = conn.create_bucket(bucket_name) 2023-09-06T18:54:40.084 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 619, in create_bucket 2023-09-06T18:54:40.084 INFO:teuthology.orchestra.run.smithi099.stderr: data=data) 2023-09-06T18:54:40.084 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/s3/connection.py", line 671, in make_request 2023-09-06T18:54:40.084 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler 2023-09-06T18:54:40.084 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1071, in make_request 2023-09-06T18:54:40.085 INFO:teuthology.orchestra.run.smithi099.stderr: retry_handler=retry_handler) 2023-09-06T18:54:40.085 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 1030, in _mexe 2023-09-06T18:54:40.085 INFO:teuthology.orchestra.run.smithi099.stderr: raise ex 2023-09-06T18:54:40.085 INFO:teuthology.orchestra.run.smithi099.stderr: File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto/connection.py", line 943, in _mexe 2023-09-06T18:54:40.086 INFO:teuthology.orchestra.run.smithi099.stderr: request.body, request.headers) 2023-09-06T18:54:40.086 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1273, in request 2023-09-06T18:54:40.086 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_request(method, url, body, headers, encode_chunked) 2023-09-06T18:54:40.086 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request 2023-09-06T18:54:40.086 INFO:teuthology.orchestra.run.smithi099.stderr: self.endheaders(body, encode_chunked=encode_chunked) 2023-09-06T18:54:40.087 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders 2023-09-06T18:54:40.087 INFO:teuthology.orchestra.run.smithi099.stderr: self._send_output(message_body, encode_chunked=encode_chunked) 2023-09-06T18:54:40.087 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 1044, in _send_output 2023-09-06T18:54:40.087 INFO:teuthology.orchestra.run.smithi099.stderr: self.send(msg) 2023-09-06T18:54:40.087 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 982, in send 2023-09-06T18:54:40.088 INFO:teuthology.orchestra.run.smithi099.stderr: self.connect() 2023-09-06T18:54:40.088 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/http/client.py", line 954, in connect 2023-09-06T18:54:40.088 INFO:teuthology.orchestra.run.smithi099.stderr: (self.host,self.port), self.timeout, self.source_address) 2023-09-06T18:54:40.088 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 724, in create_connection 2023-09-06T18:54:40.088 INFO:teuthology.orchestra.run.smithi099.stderr: raise err 2023-09-06T18:54:40.088 INFO:teuthology.orchestra.run.smithi099.stderr: File "/usr/lib64/python3.6/socket.py", line 713, in create_connection 2023-09-06T18:54:40.089 INFO:teuthology.orchestra.run.smithi099.stderr: sock.connect(sa) 2023-09-06T18:54:40.089 INFO:teuthology.orchestra.run.smithi099.stderr:ConnectionRefusedError: [Errno 111] Connection refused 2023-09-06T18:54:40.089 INFO:teuthology.orchestra.run.smithi099.stderr:-------------------- >> begin captured logging << -------------------- 2023-09-06T18:54:40.089 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using access key provided by client. 2023-09-06T18:54:40.090 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Using secret key provided by client. 2023-09-06T18:54:40.090 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: path=/cjhpse-12/ 2023-09-06T18:54:40.090 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: auth_path=/cjhpse-12/ 2023-09-06T18:54:40.090 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Method: PUT 2023-09-06T18:54:40.090 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Path: /cjhpse-12/ 2023-09-06T18:54:40.091 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Data: 2023-09-06T18:54:40.091 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Headers: {} 2023-09-06T18:54:40.091 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Host: smithi099.front.sepia.ceph.com 2023-09-06T18:54:40.091 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Port: 80 2023-09-06T18:54:40.091 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Params: {} 2023-09-06T18:54:40.092 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.092 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.092 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.092 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.092 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.093 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.093 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:54:01 GMT 2023-09-06T18:54:40.093 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-12/ 2023-09-06T18:54:40.093 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.093 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:zqBA1woPVjICqw/S7fGbYtPheM8= 2023-09-06T18:54:40.093 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:54:01 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:zqBA1woPVjICqw/S7fGbYtPheM8=', 'Content-Length': '0'} 2023-09-06T18:54:40.094 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.094 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.094 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.094 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.095 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.095 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.095 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.095 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:54:01 GMT 2023-09-06T18:54:40.095 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-12/ 2023-09-06T18:54:40.095 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.096 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:zqBA1woPVjICqw/S7fGbYtPheM8= 2023-09-06T18:54:40.096 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:54:01 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:zqBA1woPVjICqw/S7fGbYtPheM8=', 'Content-Length': '0'} 2023-09-06T18:54:40.096 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.096 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.096 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.097 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.097 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.097 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.097 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.097 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:54:01 GMT 2023-09-06T18:54:40.098 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-12/ 2023-09-06T18:54:40.098 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.098 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:zqBA1woPVjICqw/S7fGbYtPheM8= 2023-09-06T18:54:40.098 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:54:01 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:zqBA1woPVjICqw/S7fGbYtPheM8=', 'Content-Length': '0'} 2023-09-06T18:54:40.098 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.099 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.099 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.099 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.099 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.099 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.100 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.100 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:54:01 GMT 2023-09-06T18:54:40.100 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-12/ 2023-09-06T18:54:40.100 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.100 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:zqBA1woPVjICqw/S7fGbYtPheM8= 2023-09-06T18:54:40.100 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:54:01 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:zqBA1woPVjICqw/S7fGbYtPheM8=', 'Content-Length': '0'} 2023-09-06T18:54:40.101 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.101 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.101 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.101 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.101 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.101 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.102 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.102 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:54:01 GMT 2023-09-06T18:54:40.102 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-12/ 2023-09-06T18:54:40.102 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.102 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:zqBA1woPVjICqw/S7fGbYtPheM8= 2023-09-06T18:54:40.103 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:54:01 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:zqBA1woPVjICqw/S7fGbYtPheM8=', 'Content-Length': '0'} 2023-09-06T18:54:40.103 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.103 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.103 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:40.103 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:40.104 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.104 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.104 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(11, >) 2023-09-06T18:54:40.104 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(11, _) updated handler RdWr 2023-09-06T18:54:40.104 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:40.104 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1908.158544946 and callback=>; now=1878.158544946; delay=30.0 2023-09-06T18:54:40.105 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(11) 2023-09-06T18:54:40.105 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(11) updated handler Rd 2023-09-06T18:54:40.105 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:40.105 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received heartbeat frame 2023-09-06T18:54:40.105 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Recv would block on 2023-09-06T18:54:40.106 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.106 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Sending heartbeat frame 2023-09-06T18:54:40.106 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.set_writer(15, >) 2023-09-06T18:54:40.106 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: set_writer(15, _) updated handler RdWr 2023-09-06T18:54:40.106 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned on writability watcher: 2023-09-06T18:54:40.106 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1908.17040732 and callback=>; now=1878.17040732; delay=30.0 2023-09-06T18:54:40.107 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: SelectorIOServicesAdapter.remove_writer(15) 2023-09-06T18:54:40.107 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.selector_ioloop_adapter: DEBUG: remove_writer(15) updated handler Rd 2023-09-06T18:54:40.107 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.utils.io_services_utils: DEBUG: Turned off writability watcher: 2023-09-06T18:54:40.107 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 26 heartbeat frames, sent 26, idle intervals 0 2023-09-06T18:54:40.107 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1943.178068318 and callback=>; now=1878.178068318; delay=65 2023-09-06T18:54:40.108 INFO:teuthology.orchestra.run.smithi099.stderr:pika.heartbeat: DEBUG: Received 26 heartbeat frames, sent 26, idle intervals 0 2023-09-06T18:54:40.108 INFO:teuthology.orchestra.run.smithi099.stderr:pika.adapters.select_connection: DEBUG: call_later: added timeout with deadline=1943.189877397 and callback=>; now=1878.189877397; delay=65 2023-09-06T18:54:40.108 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.108 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.108 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.108 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.109 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.109 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:54:01 GMT 2023-09-06T18:54:40.109 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-12/ 2023-09-06T18:54:40.109 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.109 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:zqBA1woPVjICqw/S7fGbYtPheM8= 2023-09-06T18:54:40.109 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:54:01 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:zqBA1woPVjICqw/S7fGbYtPheM8=', 'Content-Length': '0'} 2023-09-06T18:54:40.110 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.110 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.110 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Token: None 2023-09-06T18:54:40.110 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: StringToSign: 2023-09-06T18:54:40.110 INFO:teuthology.orchestra.run.smithi099.stderr:PUT 2023-09-06T18:54:40.111 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.111 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.111 INFO:teuthology.orchestra.run.smithi099.stderr:Wed, 06 Sep 2023 18:54:01 GMT 2023-09-06T18:54:40.111 INFO:teuthology.orchestra.run.smithi099.stderr:/cjhpse-12/ 2023-09-06T18:54:40.111 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Signature: 2023-09-06T18:54:40.111 INFO:teuthology.orchestra.run.smithi099.stderr:AWS DPCRSOKICJWGAVMTJRSS:zqBA1woPVjICqw/S7fGbYtPheM8= 2023-09-06T18:54:40.112 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.8 Linux/4.18.0-513.el8.x86_64', 'Date': 'Wed, 06 Sep 2023 18:54:01 GMT', 'Authorization': 'AWS DPCRSOKICJWGAVMTJRSS:zqBA1woPVjICqw/S7fGbYtPheM8=', 'Content-Length': '0'} 2023-09-06T18:54:40.112 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: encountered ConnectionRefusedError exception, reconnecting 2023-09-06T18:54:40.112 INFO:teuthology.orchestra.run.smithi099.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 80} 2023-09-06T18:54:40.112 INFO:teuthology.orchestra.run.smithi099.stderr:--------------------- >> end captured logging << --------------------- 2023-09-06T18:54:40.112 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.113 INFO:teuthology.orchestra.run.smithi099.stderr:---------------------------------------------------------------------- 2023-09-06T18:54:40.113 INFO:teuthology.orchestra.run.smithi099.stderr:Ran 12 tests in 817.054s 2023-09-06T18:54:40.113 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T18:54:40.113 INFO:teuthology.orchestra.run.smithi099.stderr:FAILED (errors=11) 2023-09-06T18:54:40.114 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/notification_tests.py", line 238, in run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (bucket notification tests against different endpoints) on smithi099 with status 1: 'BNTESTS_CONF=/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/bn-tests.client.0.conf /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/bin/python -m nose -s /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py -v -a amqp_test' 2023-09-06T18:54:40.114 INFO:tasks.notification_tests:Removing bn-tests.conf file... 2023-09-06T18:54:40.115 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/bn-tests.client.0.conf 2023-09-06T18:54:40.139 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph 2023-09-06T18:54:40.229 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setuser ceph since I am not root 2023-09-06T18:54:40.230 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setgroup ceph since I am not root 2023-09-06T18:59:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:failed to fetch mon config (--no-mon-config to skip) 2023-09-06T18:59:40.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-06T18:59:40.234 INFO:tasks.notification_tests:Pre-processing completed... 2023-09-06T18:59:40.236 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/.aws/models/s3/2006-03-01/service-2.sdk-extras.json 2023-09-06T18:59:40.258 DEBUG:teuthology.orchestra.run.smithi099:> cd /home/ubuntu/ && rmdir -p .aws/models/s3/2006-03-01/ 2023-09-06T18:59:40.338 INFO:tasks.notification_tests:Removing bucket-notifications-tests... 2023-09-06T18:59:40.339 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/ceph 2023-09-06T18:59:40.711 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/notification_tests.py", line 151, in create_users yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/notification_tests.py", line 238, in run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (bucket notification tests against different endpoints) on smithi099 with status 1: 'BNTESTS_CONF=/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/bn-tests.client.0.conf /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/bin/python -m nose -s /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py -v -a amqp_test' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/notification_tests.py", line 306, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/notification_tests.py", line 45, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/notification_tests.py", line 158, in create_users ctx.cluster.only(client).run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi099 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph' 2023-09-06T18:59:40.783 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=dc31b78c3cc549e098f6368033a6beac Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/notification_tests.py", line 151, in create_users yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/notification_tests.py", line 238, in run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (bucket notification tests against different endpoints) on smithi099 with status 1: 'BNTESTS_CONF=/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/bn-tests.client.0.conf /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/bin/python -m nose -s /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py -v -a amqp_test' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/notification_tests.py", line 306, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/notification_tests.py", line 45, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/notification_tests.py", line 158, in create_users ctx.cluster.only(client).run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi099 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph' 2023-09-06T18:59:40.788 DEBUG:teuthology.run_tasks:Unwinding manager notification-tests 2023-09-06T18:59:40.797 DEBUG:teuthology.run_tasks:Unwinding manager rabbitmq 2023-09-06T18:59:40.814 INFO:tasks.rabbitmq:Stopping RabbitMQ Service... 2023-09-06T18:59:40.815 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/service rabbitmq-server stop 2023-09-06T18:59:40.860 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl stop rabbitmq-server.service 2023-09-06T18:59:43.400 INFO:tasks.rabbitmq:Removing packaged dependencies of RabbitMQ... 2023-09-06T18:59:43.401 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y remove rabbitmq-server.noarch 2023-09-06T18:59:43.940 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-06T18:59:43.941 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:59:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-06T18:59:43.941 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:59:43.941 INFO:teuthology.orchestra.run.smithi099.stdout:Removing: 2023-09-06T18:59:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: rabbitmq-server noarch 3.12.4-1.el8 @rabbitmq_rabbitmq-server 22 M 2023-09-06T18:59:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:59:43.942 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-06T18:59:43.943 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-06T18:59:43.943 INFO:teuthology.orchestra.run.smithi099.stdout:Remove 1 Package 2023-09-06T18:59:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:59:43.943 INFO:teuthology.orchestra.run.smithi099.stdout:Freed space: 22 M 2023-09-06T18:59:43.944 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-06T18:59:43.946 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-06T18:59:43.946 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-06T18:59:43.973 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-06T18:59:43.974 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-06T18:59:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-06T18:59:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rabbitmq-server-3.12.4-1.el8.noarch 1/1 2023-09-06T18:59:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: Erasing : rabbitmq-server-3.12.4-1.el8.noarch 1/1 2023-09-06T18:59:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rabbitmq-server-3.12.4-1.el8.noarch 1/1 2023-09-06T18:59:45.036 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rabbitmq-server-3.12.4-1.el8.noarch 1/1 2023-09-06T18:59:45.036 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:59:45.036 INFO:teuthology.orchestra.run.smithi099.stdout:Removed: 2023-09-06T18:59:45.037 INFO:teuthology.orchestra.run.smithi099.stdout: rabbitmq-server-3.12.4-1.el8.noarch 2023-09-06T18:59:45.037 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-06T18:59:45.037 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-06T18:59:45.321 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2023-09-06T18:59:45.331 DEBUG:tasks.rgw.client.0:waiting for process to exit 2023-09-06T18:59:45.331 INFO:teuthology.orchestra.run:waiting for 300 2023-09-06T18:59:45.332 INFO:tasks.rgw.client.0:Stopped 2023-09-06T18:59:45.332 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2023-09-06T18:59:45.351 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/vault-root-token 2023-09-06T18:59:45.420 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process', '--include-all'] 2023-09-06T18:59:45.420 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process', '--include-all'] 2023-09-06T18:59:45.421 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all 2023-09-06T18:59:45.505 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setuser ceph since I am not root 2023-09-06T18:59:45.505 INFO:teuthology.orchestra.run.smithi099.stderr:ignoring --setgroup ceph since I am not root 2023-09-06T19:04:45.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-06T19:04:45.505+0000 7f7f0130e780 0 monclient(hunting): authenticate timed out after 300 2023-09-06T19:04:45.508 INFO:teuthology.orchestra.run.smithi099.stderr:failed to fetch mon config (--no-mon-config to skip) 2023-09-06T19:04:45.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-06T19:04:45.510 ERROR:teuthology.run_tasks:Manager failed: rgw Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 473, in task yield File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 300, in create_pools yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 242, in start_rgw rgwadmin(ctx, client, cmd=['gc', 'process', '--include-all'], check_status=True) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/util/rgw.py", line 34, in rgwadmin proc = remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi099 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all' 2023-09-06T19:04:45.518 DEBUG:teuthology.run_tasks:Unwinding manager openssl_keys 2023-09-06T19:04:45.532 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-09-06T19:04:45.547 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-06T19:04:45.548 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-06T19:06:45.597 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-09-06T19:06:45.599 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 473, in task yield File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 300, in create_pools yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 242, in start_rgw rgwadmin(ctx, client, cmd=['gc', 'process', '--include-all'], check_status=True) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/util/rgw.py", line 34, in rgwadmin proc = remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi099 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi099 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-06T19:06:45.618 INFO:teuthology.misc:Shutting down mds daemons... 2023-09-06T19:06:45.618 INFO:teuthology.misc:Shutting down osd daemons... 2023-09-06T19:06:45.619 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-09-06T19:06:45.620 INFO:teuthology.orchestra.run:waiting for 300 2023-09-06T19:06:45.625 INFO:tasks.ceph.osd.0:Stopped 2023-09-06T19:06:45.625 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-09-06T19:06:45.626 INFO:teuthology.orchestra.run:waiting for 300 2023-09-06T19:06:45.627 INFO:tasks.ceph.osd.1:Stopped 2023-09-06T19:06:45.627 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-09-06T19:06:45.628 INFO:teuthology.orchestra.run:waiting for 300 2023-09-06T19:06:45.629 INFO:tasks.ceph.osd.2:Stopped 2023-09-06T19:06:45.629 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-09-06T19:06:45.630 INFO:teuthology.orchestra.run:waiting for 300 2023-09-06T19:06:45.631 INFO:tasks.ceph.osd.3:Stopped 2023-09-06T19:06:45.631 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-09-06T19:06:45.632 INFO:teuthology.orchestra.run:waiting for 300 2023-09-06T19:06:45.632 INFO:tasks.ceph.osd.4:Stopped 2023-09-06T19:06:45.633 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-09-06T19:06:45.634 INFO:teuthology.orchestra.run:waiting for 300 2023-09-06T19:06:45.634 INFO:tasks.ceph.osd.5:Stopped 2023-09-06T19:06:45.635 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-09-06T19:06:45.635 INFO:teuthology.orchestra.run:waiting for 300 2023-09-06T19:06:45.636 INFO:tasks.ceph.osd.6:Stopped 2023-09-06T19:06:45.637 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-09-06T19:06:45.637 INFO:teuthology.orchestra.run:waiting for 300 2023-09-06T19:06:45.638 INFO:tasks.ceph.osd.7:Stopped 2023-09-06T19:06:45.639 INFO:teuthology.misc:Shutting down mgr daemons... 2023-09-06T19:06:45.639 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-09-06T19:06:45.640 INFO:teuthology.orchestra.run:waiting for 300 2023-09-06T19:06:45.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-06T19:06:45.641 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 473, in task yield File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 300, in create_pools yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 242, in start_rgw rgwadmin(ctx, client, cmd=['gc', 'process', '--include-all'], check_status=True) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/util/rgw.py", line 34, in rgwadmin proc = remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi099 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi099 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi099 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y' 2023-09-06T19:06:45.658 INFO:tasks.ceph.mgr.y:Stopped 2023-09-06T19:06:45.659 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-06T19:06:45.660 INFO:teuthology.orchestra.run:waiting for 300 2023-09-06T19:06:45.660 INFO:tasks.ceph.mgr.x:Stopped 2023-09-06T19:06:45.661 INFO:teuthology.misc:Shutting down mon daemons... 2023-09-06T19:06:45.662 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-09-06T19:06:45.662 INFO:teuthology.orchestra.run:waiting for 300 2023-09-06T19:06:45.663 INFO:tasks.ceph.mon.a:Stopped 2023-09-06T19:06:45.664 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-09-06T19:06:45.664 INFO:teuthology.orchestra.run:waiting for 300 2023-09-06T19:06:45.665 INFO:tasks.ceph.mon.c:Stopped 2023-09-06T19:06:45.665 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-09-06T19:06:45.666 INFO:teuthology.orchestra.run:waiting for 300 2023-09-06T19:06:45.667 INFO:tasks.ceph.mon.b:Stopped 2023-09-06T19:06:45.667 INFO:tasks.ceph:Checking cluster log for badness... 2023-09-06T19:06:45.668 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | head -n 1 2023-09-06T19:06:45.708 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi099.front.sepia.ceph.com 2023-09-06T19:06:45.709 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-09-06T19:06:45.828 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi099.front.sepia.ceph.com 2023-09-06T19:06:45.829 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-09-06T19:06:45.916 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi099.front.sepia.ceph.com 2023-09-06T19:06:45.917 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-09-06T19:06:46.006 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi099.front.sepia.ceph.com 2023-09-06T19:06:46.007 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-09-06T19:06:46.083 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi181.front.sepia.ceph.com 2023-09-06T19:06:46.084 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-09-06T19:06:46.195 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi181.front.sepia.ceph.com 2023-09-06T19:06:46.195 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-09-06T19:06:46.257 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi181.front.sepia.ceph.com 2023-09-06T19:06:46.258 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-09-06T19:06:46.329 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi181.front.sepia.ceph.com 2023-09-06T19:06:46.330 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-09-06T19:06:46.407 INFO:tasks.ceph:Archiving mon data... 2023-09-06T19:06:46.410 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243/data/mon.a.tgz 2023-09-06T19:06:46.410 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2023-09-06T19:06:46.430 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.k86e2yRebP 2023-09-06T19:06:46.431 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.k86e2yRebP 2023-09-06T19:06:46.581 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.k86e2yRebP 2023-09-06T19:06:46.669 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.k86e2yRebP is 363KB 2023-09-06T19:06:46.765 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.k86e2yRebP 2023-09-06T19:06:46.783 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243/data/mon.c.tgz 2023-09-06T19:06:46.784 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2023-09-06T19:06:46.840 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.VWfFYASoUX 2023-09-06T19:06:46.840 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.VWfFYASoUX 2023-09-06T19:06:47.013 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.VWfFYASoUX 2023-09-06T19:06:47.101 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.VWfFYASoUX is 489KB 2023-09-06T19:06:47.198 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.VWfFYASoUX 2023-09-06T19:06:47.216 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243/data/mon.b.tgz 2023-09-06T19:06:47.217 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2023-09-06T19:06:47.240 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.kN5JGjSFOm 2023-09-06T19:06:47.240 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.kN5JGjSFOm 2023-09-06T19:06:47.438 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.kN5JGjSFOm 2023-09-06T19:06:47.527 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.kN5JGjSFOm is 489KB 2023-09-06T19:06:47.627 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.kN5JGjSFOm 2023-09-06T19:06:47.646 INFO:tasks.ceph:Cleaning ceph cluster... 2023-09-06T19:06:47.647 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-06T19:06:47.649 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-06T19:06:47.712 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-09-06T19:06:47.713 DEBUG:teuthology.orchestra.run.smithi099:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-06T19:06:47.718 DEBUG:teuthology.orchestra.run.smithi181:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-06T19:06:47.744 INFO:teuthology.orchestra.run.smithi099.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-06T19:06:47.780 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-06T19:06:47.786 INFO:tasks.ceph:Archiving crash dumps... 2023-09-06T19:06:47.788 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/crash to /home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243/remote/smithi099/crash 2023-09-06T19:06:47.790 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-06T19:06:47.847 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/crash to /home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243/remote/smithi181/crash 2023-09-06T19:06:47.848 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-06T19:06:47.875 INFO:tasks.ceph:Compressing logs... 2023-09-06T19:06:47.876 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-06T19:06:47.879 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-06T19:06:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122292.log 2023-09-06T19:06:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123143.log 2023-09-06T19:06:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129071.log 2023-09-06T19:06:47.911 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122292.log.gz 2023-09-06T19:06:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.123143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.123618.log 2023-09-06T19:06:47.912 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123143.log.gz 2023-09-06T19:06:47.912 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.129071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.123716.log 2023-09-06T19:06:47.912 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129071.log.gz 2023-09-06T19:06:47.912 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.123618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122296.log 2023-09-06T19:06:47.913 INFO:teuthology.orchestra.run.smithi099.stderr: 43.3% -- replaced with /var/log/ceph/ceph-client.0.123618.log.gz 2023-09-06T19:06:47.913 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.123716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122659.log 2023-09-06T19:06:47.913 INFO:teuthology.orchestra.run.smithi099.stderr: 43.3% -- replaced with /var/log/ceph/ceph-client.0.123716.log.gz 2023-09-06T19:06:47.913 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121800.log 2023-09-06T19:06:47.914 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122296.log.gz 2023-09-06T19:06:47.914 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122659.log.gz 2023-09-06T19:06:47.914 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/rgw.ceph.client.0.log 2023-09-06T19:06:47.915 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.121800.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-09-06T19:06:47.915 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121800.log.gz 2023-09-06T19:06:47.915 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/rgw.ceph.client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115542.log 2023-09-06T19:06:47.915 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122950.log 2023-09-06T19:06:47.916 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115542.log.gz 2023-09-06T19:06:47.916 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-09-06T19:06:47.916 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122950.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.122950.log.gz-- 2023-09-06T19:06:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.crash.127520.log 2023-09-06T19:06:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.123814.log 2023-09-06T19:06:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.crash.127520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122065.log 2023-09-06T19:06:47.917 INFO:teuthology.orchestra.run.smithi099.stderr: 13.0% -- replaced with /var/log/ceph/ceph-client.crash.127520.log.gz 2023-09-06T19:06:47.918 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.123814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122600.log 2023-09-06T19:06:47.918 INFO:teuthology.orchestra.run.smithi099.stderr: 43.7% -- replaced with /var/log/ceph/ceph-client.0.123814.log.gz 2023-09-06T19:06:47.918 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.128934.log 2023-09-06T19:06:47.919 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122065.log.gz 2023-09-06T19:06:47.919 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.128243.log 2023-09-06T19:06:47.919 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122600.log.gz 2023-09-06T19:06:47.919 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.128934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123192.log 2023-09-06T19:06:47.920 INFO:teuthology.orchestra.run.smithi099.stderr: 8.2% -- replaced with /var/log/ceph/ceph-client.0.128934.log.gz 2023-09-06T19:06:47.920 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-09-06T19:06:47.920 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.128243.log: 6.1% -- replaced with /var/log/ceph/ceph-client.0.128243.log.gz 2023-09-06T19:06:47.921 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.123192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122608.log 2023-09-06T19:06:47.921 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123192.log.gz 2023-09-06T19:06:47.921 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122278.log 2023-09-06T19:06:47.922 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122608.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-09-06T19:06:47.922 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122608.log.gz 2023-09-06T19:06:47.922 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122666.log 2023-09-06T19:06:47.922 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122278.log.gz 2023-09-06T19:06:47.928 INFO:teuthology.orchestra.run.smithi099.stderr: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-09-06T19:06:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-09-06T19:06:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122976.log 2023-09-06T19:06:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122666.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122666.log.gz 2023-09-06T19:06:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122603.log 2023-09-06T19:06:47.930 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122976.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122976.log.gz 2023-09-06T19:06:47.930 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122603.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122603.log.gz 2023-09-06T19:06:47.930 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127533.log 2023-09-06T19:06:47.930 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122692.log 2023-09-06T19:06:47.931 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.127533.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127533.log.gz 2023-09-06T19:06:47.931 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122692.log.gz 2023-09-06T19:06:47.931 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.0.log 2023-09-06T19:06:47.931 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121918.log 2023-09-06T19:06:47.932 INFO:teuthology.orchestra.run.smithi099.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-09-06T19:06:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ops-log-ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121967.log 2023-09-06T19:06:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.121918.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121918.log.gz 2023-09-06T19:06:47.933 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.121967.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121967.log.gz 2023-09-06T19:06:47.933 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122949.log 2023-09-06T19:06:47.933 INFO:teuthology.orchestra.run.smithi099.stderr: 93.1% -- replaced with /var/log/ceph/ops-log-ceph-client.0.log.gz 2023-09-06T19:06:47.933 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122610.log 2023-09-06T19:06:47.934 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116045.log 2023-09-06T19:06:47.934 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122949.log.gz 2023-09-06T19:06:47.934 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122610.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122610.log.gz 2023-09-06T19:06:47.935 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121869.log 2023-09-06T19:06:47.935 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.116045.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116045.log.gz 2023-09-06T19:06:47.935 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.121869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121869.log.gz 2023-09-06T19:06:47.935 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125467.log 2023-09-06T19:06:47.936 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123239.log 2023-09-06T19:06:47.936 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.125467.log: 82.6% -- replaced with /var/log/ceph/ceph-client.0.125467.log.gz 2023-09-06T19:06:47.936 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.123239.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123239.log.gz 2023-09-06T19:06:47.937 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.109304.log 2023-09-06T19:06:47.937 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123094.log 2023-09-06T19:06:47.937 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph.tmp-client.admin.109304.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-09-06T19:06:47.937 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.109304.log.gz 2023-09-06T19:06:47.938 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.123094.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123094.log.gz 2023-09-06T19:06:47.938 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122602.log 2023-09-06T19:06:47.938 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123384.log 2023-09-06T19:06:47.938 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122602.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122602.log.gz 2023-09-06T19:06:47.939 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.123384.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123384.log.gz 2023-09-06T19:06:47.939 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122291.log 2023-09-06T19:06:47.939 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-09-06T19:06:47.940 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122291.log.gz 2023-09-06T19:06:47.940 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-09-06T19:06:47.942 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph.log: 93.6% -- replaced with /var/log/ceph/ceph.log.gz 2023-09-06T19:06:47.943 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122286.log 2023-09-06T19:06:47.943 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122286.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122286.log.gz 2023-09-06T19:06:47.943 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123288.log 2023-09-06T19:06:47.944 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.123288.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123288.log.gz 2023-09-06T19:06:47.944 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115915.log 2023-09-06T19:06:47.944 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115915.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115915.log.gz 2023-09-06T19:06:47.944 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.123520.log 2023-09-06T19:06:47.945 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.123520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123335.log 2023-09-06T19:06:47.945 INFO:teuthology.orchestra.run.smithi099.stderr: 43.7% -- replaced with /var/log/ceph/ceph-client.0.123520.log.gz 2023-09-06T19:06:47.945 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122289.log 2023-09-06T19:06:47.946 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.123335.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123335.log.gz 2023-09-06T19:06:47.946 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122289.log.gz 2023-09-06T19:06:47.946 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122016.log 2023-09-06T19:06:47.946 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.crash.smithi099.127507.log 2023-09-06T19:06:47.947 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122016.log.gz 2023-09-06T19:06:47.947 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.crash.smithi099.127507.log: 50.4% -- replaced with /var/log/ceph/ceph-client.crash.smithi099.127507.log.gz 2023-09-06T19:06:47.947 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-09-06T19:06:47.948 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.123408.log 2023-09-06T19:06:47.948 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.123408.log: 96.0% -- replaced with /var/log/ceph/ceph-client.0.123408.log.gz 2023-09-06T19:06:47.949 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115515.log 2023-09-06T19:06:47.949 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115712.log 2023-09-06T19:06:47.951 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-09-06T19:06:47.951 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115515.log.gz 2023-09-06T19:06:47.952 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115614.log 2023-09-06T19:06:47.952 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115712.log.gz 2023-09-06T19:06:47.952 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-09-06T19:06:47.952 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.115614.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-09-06T19:06:47.953 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115614.log.gz 2023-09-06T19:06:47.953 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115664.log 2023-09-06T19:06:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122294.log 2023-09-06T19:06:47.956 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-09-06T19:06:47.956 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-09-06T19:06:47.956 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115465.log 2023-09-06T19:06:47.957 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115664.log.gz 2023-09-06T19:06:47.957 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-09-06T19:06:47.957 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115465.log.gz 2023-09-06T19:06:47.958 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-09-06T19:06:47.958 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-09-06T19:06:47.958 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph.log: /var/log/ceph/ceph-osd.4.log: 88.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-09-06T19:06:47.958 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115367.log 2023-09-06T19:06:47.959 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115416.log 2023-09-06T19:06:47.959 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122294.log.gz 2023-09-06T19:06:47.960 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122898.log 2023-09-06T19:06:47.960 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115367.log.gz 2023-09-06T19:06:47.961 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115416.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115416.log.gz 2023-09-06T19:06:47.961 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-09-06T19:06:47.961 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115565.log 2023-09-06T19:06:47.962 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.115565.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115565.log.gz 2023-09-06T19:06:47.962 INFO:teuthology.orchestra.run.smithi099.stderr: 92.8% -- replaced with /var/log/ceph/rgw.ceph.client.0.log.gz 2023-09-06T19:06:47.963 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122898.log.gz 2023-09-06T19:06:47.963 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121704.log 2023-09-06T19:06:47.964 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.121704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122295.log 2023-09-06T19:06:47.964 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121704.log.gz 2023-09-06T19:06:47.965 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.122295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122295.log.gz 2023-09-06T19:06:48.030 INFO:teuthology.orchestra.run.smithi181.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-09-06T19:06:48.074 INFO:teuthology.orchestra.run.smithi099.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-09-06T19:06:48.091 INFO:teuthology.orchestra.run.smithi181.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-09-06T19:06:48.228 INFO:teuthology.orchestra.run.smithi099.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-09-06T19:06:48.349 INFO:teuthology.orchestra.run.smithi181.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-09-06T19:06:48.406 INFO:teuthology.orchestra.run.smithi099.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-09-06T19:06:48.495 INFO:teuthology.orchestra.run.smithi099.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-09-06T19:06:48.507 INFO:teuthology.orchestra.run.smithi181.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-09-06T19:06:48.518 INFO:teuthology.orchestra.run.smithi181.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-09-06T19:06:48.522 INFO:teuthology.orchestra.run.smithi099.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-09-06T19:06:48.534 INFO:teuthology.orchestra.run.smithi099.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-09-06T19:06:48.535 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-06T19:06:48.536 INFO:teuthology.orchestra.run.smithi099.stderr:real 0m0.641s 2023-09-06T19:06:48.536 INFO:teuthology.orchestra.run.smithi099.stderr:user 0m2.751s 2023-09-06T19:06:48.536 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m0.126s 2023-09-06T19:06:48.565 INFO:teuthology.orchestra.run.smithi181.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-09-06T19:06:48.567 INFO:teuthology.orchestra.run.smithi181.stderr: 2023-09-06T19:06:48.567 INFO:teuthology.orchestra.run.smithi181.stderr:real 0m0.628s 2023-09-06T19:06:48.567 INFO:teuthology.orchestra.run.smithi181.stderr:user 0m2.267s 2023-09-06T19:06:48.568 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m0.107s 2023-09-06T19:06:48.568 INFO:tasks.ceph:Archiving logs... 2023-09-06T19:06:48.569 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243/remote/smithi099/log 2023-09-06T19:06:48.570 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-06T19:06:49.002 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243/remote/smithi181/log 2023-09-06T19:06:49.004 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-06T19:06:49.382 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 473, in task yield File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 300, in create_pools yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 242, in start_rgw rgwadmin(ctx, client, cmd=['gc', 'process', '--include-all'], check_status=True) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/util/rgw.py", line 34, in rgwadmin proc = remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi099 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi099 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-06T19:06:49.399 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-06T19:06:49.411 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 473, in task yield File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 300, in create_pools yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/rgw.py", line 242, in start_rgw rgwadmin(ctx, client, cmd=['gc', 'process', '--include-all'], check_status=True) File "/home/teuthworker/src/git.ceph.com_ceph-c_5535a92bd618b7e580a2fce32e9dfab6620d1086/qa/tasks/util/rgw.py", line 34, in rgwadmin proc = remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi099 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all' 2023-09-06T19:06:49.426 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-06T19:06:49.427 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-06T19:06:49.430 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-06T19:06:49.507 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-06T19:06:49.519 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-06T19:06:49.520 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-06T19:06:49.523 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-06T19:06:49.538 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2023-09-06T19:06:49.540 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2023-09-06T19:06:49.542 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-06T19:06:49.543 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2023-09-06T19:06:49.543 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 17 +477us[ +477us] +/- 85ms 2023-09-06T19:06:49.543 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv02.front.sepia.ceph.com 3 6 377 17 -1134us[-1128us] +/- 62ms 2023-09-06T19:06:49.543 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +799us[ +799us] +/- 73ms 2023-09-06T19:06:49.544 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-06T19:06:49.545 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-06T19:06:49.546 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-09-06T19:06:49.546 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 18 +516us[ +512us] +/- 85ms 2023-09-06T19:06:49.546 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv02.front.sepia.ceph.com 3 6 377 17 -1114us[-1119us] +/- 62ms 2023-09-06T19:06:49.547 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +826us[ +821us] +/- 73ms 2023-09-06T19:06:49.547 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-06T19:06:49.547 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-06T19:06:49.557 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-06T19:06:49.558 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-06T19:06:49.570 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-06T19:06:49.677 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-06T19:06:49.767 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-06T19:06:49.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-06T19:06:49.800 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-06T19:06:49.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-06T19:06:49.833 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-06T19:06:49.846 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-06T19:06:49.860 INFO:teuthology.task.internal:Duration was 2574.620083 seconds 2023-09-06T19:06:49.861 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-06T19:06:49.874 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-06T19:06:49.874 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-06T19:06:49.877 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-06T19:06:49.921 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-06T19:06:49.925 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-06T19:06:50.299 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-06T19:06:50.300 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2023-09-06T19:06:50.300 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-06T19:06:50.324 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2023-09-06T19:06:50.325 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-06T19:06:50.349 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-06T19:06:50.350 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-06T19:06:50.367 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-06T19:06:50.429 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-06T19:06:50.430 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-06T19:06:50.447 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-06T19:06:50.535 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-06T19:06:50.548 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-06T19:06:50.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-06T19:06:50.576 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-06T19:06:50.604 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-06T19:06:50.617 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-06T19:06:50.619 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-06T19:06:50.644 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2023-09-06T19:06:50.670 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2023-09-06T19:06:50.687 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-06T19:06:50.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-06T19:06:50.715 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-06T19:06:50.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-06T19:06:50.746 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-06T19:06:50.757 INFO:teuthology.task.internal:Transferring archived files... 2023-09-06T19:06:50.759 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243/remote/smithi099 2023-09-06T19:06:50.759 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-06T19:06:50.796 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243/remote/smithi181 2023-09-06T19:06:50.797 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-06T19:06:50.835 INFO:teuthology.task.internal:Removing archive directory... 2023-09-06T19:06:50.835 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-06T19:06:50.838 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-06T19:06:50.887 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-06T19:06:50.900 INFO:teuthology.task.internal:Not uploading archives. 2023-09-06T19:06:50.900 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-06T19:06:50.973 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-06T19:06:50.974 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-06T19:06:50.977 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-06T19:06:50.992 INFO:teuthology.orchestra.run.smithi099.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 6 19:06 /home/ubuntu/cephtest 2023-09-06T19:06:50.995 INFO:teuthology.orchestra.run.smithi181.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 6 19:06 /home/ubuntu/cephtest 2023-09-06T19:06:50.996 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-06T19:06:51.007 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-06T19:06:51.019 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi099 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all' 2023-09-06T19:06:51.020 INFO:teuthology.nuke:Checking targets against current locks 2023-09-06T19:06:51.046 DEBUG:teuthology.nuke:shortname: smithi099 2023-09-06T19:06:51.047 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-06T19:06:51.068 DEBUG:teuthology.nuke:shortname: smithi181 2023-09-06T19:06:51.068 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-06T19:06:51.073 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-06 18:13:27.778052', 'locked_by': 'scheduled_ivancich@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGgXCd8FtdPyuvBoDz1Odlgd52mKpJX9T2D8O3a4AwUgUBoffGlDGApGL+FaQmtbp7NgosmSVVFT864Bf5rUyyM='} 2023-09-06T19:06:51.090 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/ivancich-2023-09-06_18:12:39-rgw-wip-eric-testing-1-distro-default-smithi/7390243', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-06 18:13:27.776348', 'locked_by': 'scheduled_ivancich@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBxZTiiM0jxT4QB+tCztuafD3KDXgRBguwmwjNkdFGh/L+PidvLrqhDkcqEGy+D7vC5x6bcz2eMfr5qOlmg9rC4='} 2023-09-06T19:06:51.118 INFO:teuthology.orchestra.console.smithi099:Power off 2023-09-06T19:06:51.119 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-06T19:06:51.143 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off 2023-09-06T19:06:51.144 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-06T19:06:51.161 INFO:teuthology.orchestra.console.smithi181:Power off 2023-09-06T19:06:51.162 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-06T19:06:51.173 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-09-06T19:06:51.186 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2023-09-06T19:06:51.186 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-06T19:06:51.209 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-09-06T19:06:55.175 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-06T19:06:55.211 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-06T19:06:55.285 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-09-06T19:06:55.323 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-09-06T19:06:59.290 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-06T19:06:59.324 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-06T19:06:59.399 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off 2023-09-06T19:06:59.434 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-09-06T19:06:59.501 INFO:teuthology.orchestra.console.smithi099:Power off completed 2023-09-06T19:06:59.535 INFO:teuthology.orchestra.console.smithi181:Power off completed 2023-09-06T19:06:59.640 INFO:teuthology.run:Summary data: description: rgw/notifications/{beast bluestore-bitmap fixed-2 ignore-pg-availability overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}} duration: 2574.620083093643 failure_reason: 'Command failed on smithi099 with status 1: ''adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph''' flavor: default owner: scheduled_ivancich@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=dc31b78c3cc549e098f6368033a6beac status: fail success: false 2023-09-06T19:06:59.641 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-06T19:06:59.713 INFO:teuthology.run:FAIL